Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1449 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.5795. Min Act: -0.1700

Data Index: 8595150 (Open Web Text)

Max Activating Token Index: 774

Text #1

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.6165. Min Act: -0.1700

Data Index: 7118757 (Open Web Text)

Max Activating Token Index: 618

Text #2

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.5490. Min Act: -0.1700

Data Index: 6724568 (Open Web Text)

Max Activating Token Index: 119

Text #3

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.5172. Min Act: -0.1700

Data Index: 1772893 (Open Web Text)

Max Activating Token Index: 639

Text #4

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.5078. Min Act: -0.1700

Data Index: 1002055 (Open Web Text)

Max Activating Token Index: 46

Text #5

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.4461. Min Act: -0.1700

Data Index: 2767680 (Open Web Text)

Max Activating Token Index: 32

Text #6

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.4433. Min Act: -0.1700

Data Index: 2057793 (Open Web Text)

Max Activating Token Index: 60

Text #7

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3860. Min Act: -0.1700

Data Index: 3945764 (Open Web Text)

Max Activating Token Index: 358

Text #8

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3649. Min Act: -0.1700

Data Index: 8174247 (Open Web Text)

Max Activating Token Index: 559

Text #9

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.4253. Min Act: -0.1700

Data Index: 2437966 (Open Web Text)

Max Activating Token Index: 1018

Text #10

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3893. Min Act: -0.1700

Data Index: 6247166 (Open Web Text)

Max Activating Token Index: 868

Text #11

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3646. Min Act: -0.1700

Data Index: 4053669 (Open Web Text)

Max Activating Token Index: 156

Text #12

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.4085. Min Act: -0.1700

Data Index: 544328 (Open Web Text)

Max Activating Token Index: 705

Text #13

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3964. Min Act: -0.1700

Data Index: 3705381 (Open Web Text)

Max Activating Token Index: 1013

Text #14

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3748. Min Act: -0.1700

Data Index: 2600263 (Open Web Text)

Max Activating Token Index: 847

Text #15

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3867. Min Act: -0.1700

Data Index: 3278122 (Open Web Text)

Max Activating Token Index: 231

Text #16

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3574. Min Act: -0.1700

Data Index: 8085045 (Open Web Text)

Max Activating Token Index: 796

Text #17

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3978. Min Act: -0.1700

Data Index: 8174800 (Open Web Text)

Max Activating Token Index: 31

Text #18

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3784. Min Act: -0.1700

Data Index: 6183935 (Open Web Text)

Max Activating Token Index: 740

Text #19

Max Range: 4.6165. Min Range: -4.6165

Max Act: 4.3703. Min Act: -0.1700

Data Index: 2876377 (Open Web Text)

Max Activating Token Index: 765