Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2317 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.5654. Min Act: -0.1700

Data Index: 3954599 (Open Web Text)

Max Activating Token Index: 10

Text #1

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.5240. Min Act: -0.1700

Data Index: 2804214 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4958. Min Act: -0.1700

Data Index: 3333316 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4972. Min Act: -0.1700

Data Index: 5122584 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4717. Min Act: -0.1700

Data Index: 6833078 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4821. Min Act: -0.1700

Data Index: 3506548 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4537. Min Act: -0.1700

Data Index: 6815559 (Open Web Text)

Max Activating Token Index: 4

Text #7

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4450. Min Act: -0.1700

Data Index: 1201969 (Open Web Text)

Max Activating Token Index: 6

Text #8

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4403. Min Act: -0.1700

Data Index: 7091040 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4446. Min Act: -0.1700

Data Index: 7723925 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4528. Min Act: -0.1700

Data Index: 2117275 (Open Web Text)

Max Activating Token Index: 13

Text #11

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4533. Min Act: -0.1700

Data Index: 3992206 (Open Web Text)

Max Activating Token Index: 8

Text #12

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4402. Min Act: -0.1700

Data Index: 3731372 (Open Web Text)

Max Activating Token Index: 8

Text #13

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4468. Min Act: -0.1700

Data Index: 1538939 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4604. Min Act: -0.1700

Data Index: 709731 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4059. Min Act: -0.1700

Data Index: 1740189 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.3983. Min Act: -0.1700

Data Index: 1534109 (Open Web Text)

Max Activating Token Index: 18

Text #17

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4131. Min Act: -0.1700

Data Index: 5124429 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4076. Min Act: -0.1700

Data Index: 4969318 (Open Web Text)

Max Activating Token Index: 5

Text #19

Max Range: 4.5654. Min Range: -4.5654

Max Act: 4.4070. Min Act: -0.1700

Data Index: 3799543 (Open Web Text)

Max Activating Token Index: 5