Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2719 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0257. Min Range: -3.0257

Max Act: 3.0257. Min Act: -0.1700

Data Index: 6957873 (Open Web Text)

Max Activating Token Index: 88

Text #1

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9992. Min Act: -0.1700

Data Index: 8164005 (Open Web Text)

Max Activating Token Index: 134

Text #2

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9646. Min Act: -0.1700

Data Index: 8230871 (Open Web Text)

Max Activating Token Index: 353

Text #3

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9669. Min Act: -0.1700

Data Index: 4701347 (Open Web Text)

Max Activating Token Index: 314

Text #4

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9436. Min Act: -0.1700

Data Index: 2991234 (Open Web Text)

Max Activating Token Index: 182

Text #5

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9311. Min Act: -0.1700

Data Index: 2376894 (Open Web Text)

Max Activating Token Index: 10

Text #6

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9386. Min Act: -0.1700

Data Index: 939032 (Open Web Text)

Max Activating Token Index: 81

Text #7

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9274. Min Act: -0.1700

Data Index: 3508554 (Open Web Text)

Max Activating Token Index: 125

Text #8

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9089. Min Act: -0.1700

Data Index: 7657029 (Open Web Text)

Max Activating Token Index: 37

Text #9

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9011. Min Act: -0.1700

Data Index: 3706414 (Open Web Text)

Max Activating Token Index: 152

Text #10

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8990. Min Act: -0.1700

Data Index: 2652087 (Open Web Text)

Max Activating Token Index: 696

Text #11

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8878. Min Act: -0.1700

Data Index: 4648696 (Open Web Text)

Max Activating Token Index: 153

Text #12

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8927. Min Act: -0.1700

Data Index: 7560389 (Open Web Text)

Max Activating Token Index: 680

Text #13

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8988. Min Act: -0.1700

Data Index: 2049261 (Open Web Text)

Max Activating Token Index: 655

Text #14

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.9063. Min Act: -0.1700

Data Index: 1434907 (Open Web Text)

Max Activating Token Index: 116

Text #15

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8982. Min Act: -0.1700

Data Index: 6109078 (Open Web Text)

Max Activating Token Index: 32

Text #16

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8908. Min Act: -0.1700

Data Index: 3948140 (Open Web Text)

Max Activating Token Index: 58

Text #17

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8849. Min Act: -0.1700

Data Index: 4484575 (Open Web Text)

Max Activating Token Index: 961

Text #18

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8816. Min Act: -0.1700

Data Index: 6468714 (Open Web Text)

Max Activating Token Index: 577

Text #19

Max Range: 3.0257. Min Range: -3.0257

Max Act: 2.8668. Min Act: -0.1700

Data Index: 3284279 (Open Web Text)

Max Activating Token Index: 1020