Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1292 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.7106. Min Act: -0.1700

Data Index: 270659 (Open Web Text)

Max Activating Token Index: 762

Text #1

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.5828. Min Act: -0.1700

Data Index: 6234303 (Open Web Text)

Max Activating Token Index: 401

Text #2

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.5475. Min Act: -0.1700

Data Index: 4751212 (Open Web Text)

Max Activating Token Index: 940

Text #3

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.5616. Min Act: -0.1700

Data Index: 3629884 (Open Web Text)

Max Activating Token Index: 189

Text #4

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.5354. Min Act: -0.1700

Data Index: 5626988 (Open Web Text)

Max Activating Token Index: 620

Text #5

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.5396. Min Act: -0.1700

Data Index: 5374363 (Open Web Text)

Max Activating Token Index: 785

Text #6

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4950. Min Act: -0.1700

Data Index: 928533 (Open Web Text)

Max Activating Token Index: 901

Text #7

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4768. Min Act: -0.1700

Data Index: 3411703 (Open Web Text)

Max Activating Token Index: 216

Text #8

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4583. Min Act: -0.1700

Data Index: 6945335 (Open Web Text)

Max Activating Token Index: 152

Text #9

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4470. Min Act: -0.1700

Data Index: 6541653 (Open Web Text)

Max Activating Token Index: 498

Text #10

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4152. Min Act: -0.1700

Data Index: 5742200 (Open Web Text)

Max Activating Token Index: 341

Text #11

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4533. Min Act: -0.1700

Data Index: 4294811 (Open Web Text)

Max Activating Token Index: 684

Text #12

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4175. Min Act: -0.1700

Data Index: 2816341 (Open Web Text)

Max Activating Token Index: 84

Text #13

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4493. Min Act: -0.1700

Data Index: 4742479 (Open Web Text)

Max Activating Token Index: 313

Text #14

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4280. Min Act: -0.1700

Data Index: 6089834 (Open Web Text)

Max Activating Token Index: 515

Text #15

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4022. Min Act: -0.1700

Data Index: 1377277 (Open Web Text)

Max Activating Token Index: 438

Text #16

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4108. Min Act: -0.1700

Data Index: 1694885 (Open Web Text)

Max Activating Token Index: 832

Text #17

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.3817. Min Act: -0.1700

Data Index: 4957780 (Open Web Text)

Max Activating Token Index: 770

Text #18

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.4162. Min Act: -0.1700

Data Index: 4656541 (Open Web Text)

Max Activating Token Index: 382

Text #19

Max Range: 4.7106. Min Range: -4.7106

Max Act: 4.3916. Min Act: -0.1700

Data Index: 2100545 (Open Web Text)

Max Activating Token Index: 802