Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2959 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.4201. Min Act: -0.1700

Data Index: 6131569 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.4087. Min Act: -0.1700

Data Index: 6864214 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3980. Min Act: -0.1700

Data Index: 6448356 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3703. Min Act: -0.1700

Data Index: 5073118 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3564. Min Act: -0.1700

Data Index: 3655469 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3378. Min Act: -0.1700

Data Index: 4296443 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3305. Min Act: -0.1700

Data Index: 7814042 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3260. Min Act: -0.1700

Data Index: 7538025 (Open Web Text)

Max Activating Token Index: 14

Text #8

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3506. Min Act: -0.1700

Data Index: 255740 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3514. Min Act: -0.1700

Data Index: 7817917 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3416. Min Act: -0.1700

Data Index: 4232569 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3161. Min Act: -0.1700

Data Index: 5360586 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3175. Min Act: -0.1700

Data Index: 5045109 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3039. Min Act: -0.1700

Data Index: 7234518 (Open Web Text)

Max Activating Token Index: 2

Text #14

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3025. Min Act: -0.1700

Data Index: 2465278 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3051. Min Act: -0.1700

Data Index: 3529928 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3216. Min Act: -0.1700

Data Index: 73269 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3089. Min Act: -0.1700

Data Index: 2232171 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3157. Min Act: -0.1700

Data Index: 5979301 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 3.4201. Min Range: -3.4201

Max Act: 3.3238. Min Act: -0.1700

Data Index: 7554280 (Open Web Text)

Max Activating Token Index: 3