Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2247 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8815. Min Act: -0.1700

Data Index: 3562108 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.9062. Min Act: -0.1700

Data Index: 310513 (Open Web Text)

Max Activating Token Index: 494

Text #2

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8768. Min Act: -0.1700

Data Index: 5686043 (Open Web Text)

Max Activating Token Index: 378

Text #3

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8361. Min Act: -0.1700

Data Index: 7239594 (Open Web Text)

Max Activating Token Index: 430

Text #4

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8131. Min Act: -0.1700

Data Index: 7176721 (Open Web Text)

Max Activating Token Index: 982

Text #5

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8302. Min Act: -0.1700

Data Index: 3786114 (Open Web Text)

Max Activating Token Index: 113

Text #6

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8348. Min Act: -0.1700

Data Index: 6796738 (Open Web Text)

Max Activating Token Index: 689

Text #7

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8464. Min Act: -0.1700

Data Index: 4893430 (Open Web Text)

Max Activating Token Index: 15

Text #8

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.8056. Min Act: -0.1700

Data Index: 111919 (Open Web Text)

Max Activating Token Index: 115

Text #9

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7864. Min Act: -0.1700

Data Index: 4504005 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7942. Min Act: -0.1700

Data Index: 2555621 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7711. Min Act: -0.1700

Data Index: 6617176 (Open Web Text)

Max Activating Token Index: 453

Text #12

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7823. Min Act: -0.1700

Data Index: 7644227 (Open Web Text)

Max Activating Token Index: 576

Text #13

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7562. Min Act: -0.1700

Data Index: 6870080 (Open Web Text)

Max Activating Token Index: 492

Text #14

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7596. Min Act: -0.1700

Data Index: 5008550 (Open Web Text)

Max Activating Token Index: 526

Text #15

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7379. Min Act: -0.1700

Data Index: 6412285 (Open Web Text)

Max Activating Token Index: 196

Text #16

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7168. Min Act: -0.1700

Data Index: 6659407 (Open Web Text)

Max Activating Token Index: 869

Text #17

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7381. Min Act: -0.1700

Data Index: 1417358 (Open Web Text)

Max Activating Token Index: 107

Text #18

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7453. Min Act: -0.1700

Data Index: 6665707 (Open Web Text)

Max Activating Token Index: 952

Text #19

Max Range: 3.9062. Min Range: -3.9062

Max Act: 3.7293. Min Act: -0.1700

Data Index: 4898295 (Open Web Text)

Max Activating Token Index: 145