Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3007 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8519. Min Act: -0.1700

Data Index: 1676341 (Open Web Text)

Max Activating Token Index: 56

Text #1

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8419. Min Act: -0.1700

Data Index: 4681698 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8430. Min Act: -0.1700

Data Index: 2791931 (Open Web Text)

Max Activating Token Index: 49

Text #3

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8227. Min Act: -0.1700

Data Index: 8223167 (Open Web Text)

Max Activating Token Index: 799

Text #4

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8033. Min Act: -0.1700

Data Index: 8222829 (Open Web Text)

Max Activating Token Index: 732

Text #5

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.8004. Min Act: -0.1700

Data Index: 85648 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7744. Min Act: -0.1700

Data Index: 1650949 (Open Web Text)

Max Activating Token Index: 68

Text #7

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7701. Min Act: -0.1700

Data Index: 7366401 (Open Web Text)

Max Activating Token Index: 947

Text #8

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7720. Min Act: -0.1700

Data Index: 2424197 (Open Web Text)

Max Activating Token Index: 397

Text #9

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7715. Min Act: -0.1700

Data Index: 4585643 (Open Web Text)

Max Activating Token Index: 1007

Text #10

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7691. Min Act: -0.1700

Data Index: 3280909 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7729. Min Act: -0.1700

Data Index: 6417988 (Open Web Text)

Max Activating Token Index: 224

Text #12

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7687. Min Act: -0.1700

Data Index: 5695464 (Open Web Text)

Max Activating Token Index: 1008

Text #13

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7690. Min Act: -0.1700

Data Index: 8413014 (Open Web Text)

Max Activating Token Index: 11

Text #14

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7622. Min Act: -0.1700

Data Index: 6845892 (Open Web Text)

Max Activating Token Index: 971

Text #15

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7580. Min Act: -0.1700

Data Index: 7081406 (Open Web Text)

Max Activating Token Index: 965

Text #16

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7581. Min Act: -0.1700

Data Index: 5592642 (Open Web Text)

Max Activating Token Index: 927

Text #17

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7510. Min Act: -0.1700

Data Index: 5366288 (Open Web Text)

Max Activating Token Index: 983

Text #18

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7651. Min Act: -0.1700

Data Index: 4876346 (Open Web Text)

Max Activating Token Index: 77

Text #19

Max Range: 2.8519. Min Range: -2.8519

Max Act: 2.7397. Min Act: -0.1700

Data Index: 7353081 (Open Web Text)

Max Activating Token Index: 4