Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 396 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.4121. Min Act: -0.1700

Data Index: 2723445 (Open Web Text)

Max Activating Token Index: 477

Text #1

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.3963. Min Act: -0.1700

Data Index: 4441219 (Open Web Text)

Max Activating Token Index: 664

Text #2

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.3262. Min Act: -0.1700

Data Index: 5488588 (Open Web Text)

Max Activating Token Index: 109

Text #3

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.3227. Min Act: -0.1700

Data Index: 7114614 (Open Web Text)

Max Activating Token Index: 552

Text #4

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2765. Min Act: -0.1700

Data Index: 2278563 (Open Web Text)

Max Activating Token Index: 364

Text #5

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2738. Min Act: -0.1700

Data Index: 1995820 (Open Web Text)

Max Activating Token Index: 103

Text #6

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2752. Min Act: -0.1700

Data Index: 4075693 (Open Web Text)

Max Activating Token Index: 41

Text #7

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2674. Min Act: -0.1700

Data Index: 4116584 (Open Web Text)

Max Activating Token Index: 114

Text #8

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2685. Min Act: -0.1700

Data Index: 4402045 (Open Web Text)

Max Activating Token Index: 447

Text #9

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2632. Min Act: -0.1700

Data Index: 7386531 (Open Web Text)

Max Activating Token Index: 290

Text #10

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2610. Min Act: -0.1700

Data Index: 8365083 (Open Web Text)

Max Activating Token Index: 31

Text #11

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2537. Min Act: -0.1700

Data Index: 8588677 (Open Web Text)

Max Activating Token Index: 318

Text #12

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2494. Min Act: -0.1700

Data Index: 3408291 (Open Web Text)

Max Activating Token Index: 518

Text #13

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2322. Min Act: -0.1700

Data Index: 725485 (Open Web Text)

Max Activating Token Index: 78

Text #14

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2295. Min Act: -0.1700

Data Index: 2724894 (Open Web Text)

Max Activating Token Index: 415

Text #15

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2291. Min Act: -0.1700

Data Index: 8612656 (Open Web Text)

Max Activating Token Index: 314

Text #16

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2391. Min Act: -0.1700

Data Index: 5863944 (Open Web Text)

Max Activating Token Index: 300

Text #17

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2353. Min Act: -0.1700

Data Index: 2805004 (Open Web Text)

Max Activating Token Index: 406

Text #18

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2440. Min Act: -0.1700

Data Index: 7956968 (Open Web Text)

Max Activating Token Index: 344

Text #19

Max Range: 3.4121. Min Range: -3.4121

Max Act: 3.2201. Min Act: -0.1700

Data Index: 3357076 (Open Web Text)

Max Activating Token Index: 405