Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 461 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.7259. Min Act: -0.1700

Data Index: 6925372 (Open Web Text)

Max Activating Token Index: 351

Text #1

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.7207. Min Act: -0.1700

Data Index: 6876648 (Open Web Text)

Max Activating Token Index: 887

Text #2

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.7209. Min Act: -0.1700

Data Index: 8465311 (Open Web Text)

Max Activating Token Index: 498

Text #3

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6780. Min Act: -0.1700

Data Index: 1253430 (Open Web Text)

Max Activating Token Index: 487

Text #4

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6899. Min Act: -0.1700

Data Index: 7015582 (Open Web Text)

Max Activating Token Index: 492

Text #5

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6963. Min Act: -0.1700

Data Index: 526340 (Open Web Text)

Max Activating Token Index: 559

Text #6

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6931. Min Act: -0.1700

Data Index: 2690464 (Open Web Text)

Max Activating Token Index: 639

Text #7

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6535. Min Act: -0.1700

Data Index: 7174196 (Open Web Text)

Max Activating Token Index: 1013

Text #8

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6682. Min Act: -0.1700

Data Index: 985519 (Open Web Text)

Max Activating Token Index: 422

Text #9

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6659. Min Act: -0.1700

Data Index: 3784086 (Open Web Text)

Max Activating Token Index: 575

Text #10

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6666. Min Act: -0.1700

Data Index: 3690617 (Open Web Text)

Max Activating Token Index: 904

Text #11

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6542. Min Act: -0.1700

Data Index: 8463588 (Open Web Text)

Max Activating Token Index: 511

Text #12

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6454. Min Act: -0.1700

Data Index: 3001266 (Open Web Text)

Max Activating Token Index: 644

Text #13

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6362. Min Act: -0.1700

Data Index: 1049944 (Open Web Text)

Max Activating Token Index: 556

Text #14

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6541. Min Act: -0.1700

Data Index: 6974101 (Open Web Text)

Max Activating Token Index: 957

Text #15

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6456. Min Act: -0.1700

Data Index: 1442295 (Open Web Text)

Max Activating Token Index: 370

Text #16

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6238. Min Act: -0.1700

Data Index: 5236857 (Open Web Text)

Max Activating Token Index: 837

Text #17

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6323. Min Act: -0.1700

Data Index: 2332693 (Open Web Text)

Max Activating Token Index: 385

Text #18

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6212. Min Act: -0.1700

Data Index: 5131971 (Open Web Text)

Max Activating Token Index: 469

Text #19

Max Range: 4.7259. Min Range: -4.7259

Max Act: 4.6152. Min Act: -0.1700

Data Index: 5137814 (Open Web Text)

Max Activating Token Index: 904