Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1251 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.5529. Min Act: -0.1700

Data Index: 41472 (Open Web Text)

Max Activating Token Index: 591

Text #1

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.5219. Min Act: -0.1700

Data Index: 3547718 (Open Web Text)

Max Activating Token Index: 354

Text #2

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.5113. Min Act: -0.1700

Data Index: 1359021 (Open Web Text)

Max Activating Token Index: 629

Text #3

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.5078. Min Act: -0.1700

Data Index: 3054241 (Open Web Text)

Max Activating Token Index: 21

Text #4

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4791. Min Act: -0.1700

Data Index: 4677028 (Open Web Text)

Max Activating Token Index: 168

Text #5

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4803. Min Act: -0.1700

Data Index: 532639 (Open Web Text)

Max Activating Token Index: 264

Text #6

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4827. Min Act: -0.1700

Data Index: 7729127 (Open Web Text)

Max Activating Token Index: 65

Text #7

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4880. Min Act: -0.1700

Data Index: 5913124 (Open Web Text)

Max Activating Token Index: 44

Text #8

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4682. Min Act: -0.1700

Data Index: 7615463 (Open Web Text)

Max Activating Token Index: 11

Text #9

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4494. Min Act: -0.1700

Data Index: 4053211 (Open Web Text)

Max Activating Token Index: 444

Text #10

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4456. Min Act: -0.1700

Data Index: 7163437 (Open Web Text)

Max Activating Token Index: 832

Text #11

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4351. Min Act: -0.1700

Data Index: 8605591 (Open Web Text)

Max Activating Token Index: 849

Text #12

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4513. Min Act: -0.1700

Data Index: 2323663 (Open Web Text)

Max Activating Token Index: 56

Text #13

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4482. Min Act: -0.1700

Data Index: 6933625 (Open Web Text)

Max Activating Token Index: 197

Text #14

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4540. Min Act: -0.1700

Data Index: 1375247 (Open Web Text)

Max Activating Token Index: 545

Text #15

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4393. Min Act: -0.1700

Data Index: 8094611 (Open Web Text)

Max Activating Token Index: 129

Text #16

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4273. Min Act: -0.1700

Data Index: 770410 (Open Web Text)

Max Activating Token Index: 635

Text #17

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4176. Min Act: -0.1700

Data Index: 4098631 (Open Web Text)

Max Activating Token Index: 530

Text #18

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4121. Min Act: -0.1700

Data Index: 80582 (Open Web Text)

Max Activating Token Index: 954

Text #19

Max Range: 3.5529. Min Range: -3.5529

Max Act: 3.4293. Min Act: -0.1700

Data Index: 4319608 (Open Web Text)

Max Activating Token Index: 71