Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 165 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0026. Min Range: -3.0026

Max Act: 3.0026. Min Act: -0.1700

Data Index: 4135542 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 3.0026. Min Range: -3.0026

Max Act: 3.0026. Min Act: -0.1700

Data Index: 8079750 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.9451. Min Act: -0.1700

Data Index: 8733982 (Open Web Text)

Max Activating Token Index: 243

Text #3

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.9128. Min Act: -0.1700

Data Index: 7755753 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 983347 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 804506 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 4557859 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 3730130 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 7871781 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 5083377 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8846. Min Act: -0.1700

Data Index: 46044 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8536. Min Act: -0.1700

Data Index: 7353470 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8559. Min Act: -0.1700

Data Index: 6475033 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8559. Min Act: -0.1700

Data Index: 6189444 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8559. Min Act: -0.1700

Data Index: 5726574 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8559. Min Act: -0.1700

Data Index: 206135 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8559. Min Act: -0.1700

Data Index: 4387366 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8371. Min Act: -0.1700

Data Index: 1940941 (Open Web Text)

Max Activating Token Index: 10

Text #18

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8190. Min Act: -0.1700

Data Index: 892537 (Open Web Text)

Max Activating Token Index: 255

Text #19

Max Range: 3.0026. Min Range: -3.0026

Max Act: 2.8157. Min Act: -0.1700

Data Index: 4119264 (Open Web Text)

Max Activating Token Index: 23