Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 752 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.5269. Min Act: -0.1700

Data Index: 1507301 (Open Web Text)

Max Activating Token Index: 578

Text #1

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.5204. Min Act: -0.1700

Data Index: 7798083 (Open Web Text)

Max Activating Token Index: 617

Text #2

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4808. Min Act: -0.1700

Data Index: 6786130 (Open Web Text)

Max Activating Token Index: 964

Text #3

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4653. Min Act: -0.1700

Data Index: 3652047 (Open Web Text)

Max Activating Token Index: 566

Text #4

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4425. Min Act: -0.1700

Data Index: 7130624 (Open Web Text)

Max Activating Token Index: 103

Text #5

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4581. Min Act: -0.1700

Data Index: 3616637 (Open Web Text)

Max Activating Token Index: 703

Text #6

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4301. Min Act: -0.1700

Data Index: 8764786 (Open Web Text)

Max Activating Token Index: 497

Text #7

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4627. Min Act: -0.1700

Data Index: 1814691 (Open Web Text)

Max Activating Token Index: 734

Text #8

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4495. Min Act: -0.1700

Data Index: 2570539 (Open Web Text)

Max Activating Token Index: 956

Text #9

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4244. Min Act: -0.1700

Data Index: 6080846 (Open Web Text)

Max Activating Token Index: 213

Text #10

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4217. Min Act: -0.1700

Data Index: 921798 (Open Web Text)

Max Activating Token Index: 761

Text #11

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4190. Min Act: -0.1700

Data Index: 1401914 (Open Web Text)

Max Activating Token Index: 510

Text #12

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4155. Min Act: -0.1700

Data Index: 5612935 (Open Web Text)

Max Activating Token Index: 501

Text #13

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4471. Min Act: -0.1700

Data Index: 8367779 (Open Web Text)

Max Activating Token Index: 622

Text #14

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4383. Min Act: -0.1700

Data Index: 2369375 (Open Web Text)

Max Activating Token Index: 972

Text #15

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4346. Min Act: -0.1700

Data Index: 658019 (Open Web Text)

Max Activating Token Index: 152

Text #16

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4128. Min Act: -0.1700

Data Index: 343995 (Open Web Text)

Max Activating Token Index: 781

Text #17

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4369. Min Act: -0.1700

Data Index: 3275593 (Open Web Text)

Max Activating Token Index: 196

Text #18

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4097. Min Act: -0.1700

Data Index: 480080 (Open Web Text)

Max Activating Token Index: 934

Text #19

Max Range: 4.5269. Min Range: -4.5269

Max Act: 4.4018. Min Act: -0.1700

Data Index: 3040475 (Open Web Text)

Max Activating Token Index: 361