Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2738 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1337. Min Range: -6.1337

Max Act: 6.1337. Min Act: -0.1700

Data Index: 8188438 (Open Web Text)

Max Activating Token Index: 968

Text #1

Max Range: 6.1337. Min Range: -6.1337

Max Act: 6.0514. Min Act: -0.1700

Data Index: 7883283 (Open Web Text)

Max Activating Token Index: 106

Text #2

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.9832. Min Act: -0.1700

Data Index: 5799711 (Open Web Text)

Max Activating Token Index: 427

Text #3

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.9790. Min Act: -0.1700

Data Index: 8245148 (Open Web Text)

Max Activating Token Index: 490

Text #4

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.9208. Min Act: -0.1700

Data Index: 8753932 (Open Web Text)

Max Activating Token Index: 515

Text #5

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.9119. Min Act: -0.1700

Data Index: 5608002 (Open Web Text)

Max Activating Token Index: 300

Text #6

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8763. Min Act: -0.1700

Data Index: 1700903 (Open Web Text)

Max Activating Token Index: 130

Text #7

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8802. Min Act: -0.1700

Data Index: 4363591 (Open Web Text)

Max Activating Token Index: 332

Text #8

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8772. Min Act: -0.1700

Data Index: 4213501 (Open Web Text)

Max Activating Token Index: 93

Text #9

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8571. Min Act: -0.1700

Data Index: 1177181 (Open Web Text)

Max Activating Token Index: 596

Text #10

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8579. Min Act: -0.1700

Data Index: 8384320 (Open Web Text)

Max Activating Token Index: 184

Text #11

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8183. Min Act: -0.1700

Data Index: 2155285 (Open Web Text)

Max Activating Token Index: 72

Text #12

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.7914. Min Act: -0.1700

Data Index: 7651430 (Open Web Text)

Max Activating Token Index: 941

Text #13

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8058. Min Act: -0.1700

Data Index: 1746376 (Open Web Text)

Max Activating Token Index: 255

Text #14

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8280. Min Act: -0.1700

Data Index: 2973732 (Open Web Text)

Max Activating Token Index: 186

Text #15

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.7996. Min Act: -0.1700

Data Index: 1962751 (Open Web Text)

Max Activating Token Index: 37

Text #16

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.7875. Min Act: -0.1700

Data Index: 6977655 (Open Web Text)

Max Activating Token Index: 62

Text #17

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.8017. Min Act: -0.1700

Data Index: 6607294 (Open Web Text)

Max Activating Token Index: 253

Text #18

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.7671. Min Act: -0.1700

Data Index: 6983717 (Open Web Text)

Max Activating Token Index: 44

Text #19

Max Range: 6.1337. Min Range: -6.1337

Max Act: 5.7802. Min Act: -0.1700

Data Index: 3174968 (Open Web Text)

Max Activating Token Index: 10