Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 196 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.4295. Min Act: -0.1700

Data Index: 2176572 (Open Web Text)

Max Activating Token Index: 287

Text #1

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.4259. Min Act: -0.1700

Data Index: 4300804 (Open Web Text)

Max Activating Token Index: 300

Text #2

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.4018. Min Act: -0.1700

Data Index: 4560817 (Open Web Text)

Max Activating Token Index: 473

Text #3

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.4377. Min Act: -0.1700

Data Index: 4560807 (Open Web Text)

Max Activating Token Index: 569

Text #4

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3932. Min Act: -0.1700

Data Index: 5271174 (Open Web Text)

Max Activating Token Index: 798

Text #5

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.4124. Min Act: -0.1700

Data Index: 1041495 (Open Web Text)

Max Activating Token Index: 281

Text #6

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3795. Min Act: -0.1700

Data Index: 7706890 (Open Web Text)

Max Activating Token Index: 41

Text #7

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3503. Min Act: -0.1700

Data Index: 8751945 (Open Web Text)

Max Activating Token Index: 818

Text #8

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3892. Min Act: -0.1700

Data Index: 7944064 (Open Web Text)

Max Activating Token Index: 90

Text #9

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3815. Min Act: -0.1700

Data Index: 3588538 (Open Web Text)

Max Activating Token Index: 519

Text #10

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3380. Min Act: -0.1700

Data Index: 5597521 (Open Web Text)

Max Activating Token Index: 305

Text #11

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3284. Min Act: -0.1700

Data Index: 8766571 (Open Web Text)

Max Activating Token Index: 636

Text #12

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3456. Min Act: -0.1700

Data Index: 3654987 (Open Web Text)

Max Activating Token Index: 520

Text #13

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3309. Min Act: -0.1700

Data Index: 3293413 (Open Web Text)

Max Activating Token Index: 466

Text #14

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3492. Min Act: -0.1700

Data Index: 4997579 (Open Web Text)

Max Activating Token Index: 982

Text #15

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3019. Min Act: -0.1700

Data Index: 452557 (Open Web Text)

Max Activating Token Index: 818

Text #16

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3169. Min Act: -0.1700

Data Index: 5569612 (Open Web Text)

Max Activating Token Index: 786

Text #17

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.2764. Min Act: -0.1700

Data Index: 3661020 (Open Web Text)

Max Activating Token Index: 626

Text #18

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3022. Min Act: -0.1700

Data Index: 8411801 (Open Web Text)

Max Activating Token Index: 77

Text #19

Max Range: 4.4377. Min Range: -4.4377

Max Act: 4.3020. Min Act: -0.1700

Data Index: 1078504 (Open Web Text)

Max Activating Token Index: 567