Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 350 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0267. Min Range: -4.0267

Max Act: 4.0267. Min Act: -0.1700

Data Index: 8491672 (Open Web Text)

Max Activating Token Index: 112

Text #1

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.9338. Min Act: -0.1700

Data Index: 867570 (Open Web Text)

Max Activating Token Index: 50

Text #2

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.9650. Min Act: -0.1700

Data Index: 1627050 (Open Web Text)

Max Activating Token Index: 298

Text #3

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.9411. Min Act: -0.1700

Data Index: 5626904 (Open Web Text)

Max Activating Token Index: 912

Text #4

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8813. Min Act: -0.1700

Data Index: 1785400 (Open Web Text)

Max Activating Token Index: 449

Text #5

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8829. Min Act: -0.1700

Data Index: 3416054 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8837. Min Act: -0.1700

Data Index: 8709856 (Open Web Text)

Max Activating Token Index: 438

Text #7

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8565. Min Act: -0.1700

Data Index: 6799764 (Open Web Text)

Max Activating Token Index: 633

Text #8

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8931. Min Act: -0.1700

Data Index: 5780255 (Open Web Text)

Max Activating Token Index: 358

Text #9

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8391. Min Act: -0.1700

Data Index: 3967153 (Open Web Text)

Max Activating Token Index: 397

Text #10

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8583. Min Act: -0.1700

Data Index: 6156167 (Open Web Text)

Max Activating Token Index: 481

Text #11

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8552. Min Act: -0.1700

Data Index: 7214352 (Open Web Text)

Max Activating Token Index: 890

Text #12

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8657. Min Act: -0.1700

Data Index: 1241607 (Open Web Text)

Max Activating Token Index: 708

Text #13

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8476. Min Act: -0.1700

Data Index: 496105 (Open Web Text)

Max Activating Token Index: 254

Text #14

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8583. Min Act: -0.1700

Data Index: 8777942 (Open Web Text)

Max Activating Token Index: 182

Text #15

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8658. Min Act: -0.1700

Data Index: 2246929 (Open Web Text)

Max Activating Token Index: 489

Text #16

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8474. Min Act: -0.1700

Data Index: 1165803 (Open Web Text)

Max Activating Token Index: 230

Text #17

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8313. Min Act: -0.1700

Data Index: 3023783 (Open Web Text)

Max Activating Token Index: 906

Text #18

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8283. Min Act: -0.1700

Data Index: 3289946 (Open Web Text)

Max Activating Token Index: 82

Text #19

Max Range: 4.0267. Min Range: -4.0267

Max Act: 3.8137. Min Act: -0.1700

Data Index: 3729171 (Open Web Text)

Max Activating Token Index: 929