Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 91 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.5235. Min Act: -0.1700

Data Index: 173728 (Open Web Text)

Max Activating Token Index: 894

Text #1

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3917. Min Act: -0.1700

Data Index: 4246195 (Open Web Text)

Max Activating Token Index: 34

Text #2

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3772. Min Act: -0.1700

Data Index: 5956813 (Open Web Text)

Max Activating Token Index: 377

Text #3

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3633. Min Act: -0.1700

Data Index: 4386728 (Open Web Text)

Max Activating Token Index: 572

Text #4

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3651. Min Act: -0.1700

Data Index: 5956812 (Open Web Text)

Max Activating Token Index: 257

Text #5

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3564. Min Act: -0.1700

Data Index: 4410681 (Open Web Text)

Max Activating Token Index: 218

Text #6

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3626. Min Act: -0.1700

Data Index: 7005882 (Open Web Text)

Max Activating Token Index: 103

Text #7

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3706. Min Act: -0.1700

Data Index: 3393752 (Open Web Text)

Max Activating Token Index: 530

Text #8

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3198. Min Act: -0.1700

Data Index: 2967604 (Open Web Text)

Max Activating Token Index: 391

Text #9

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3346. Min Act: -0.1700

Data Index: 4371742 (Open Web Text)

Max Activating Token Index: 965

Text #10

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3192. Min Act: -0.1700

Data Index: 7344638 (Open Web Text)

Max Activating Token Index: 255

Text #11

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3104. Min Act: -0.1700

Data Index: 1163382 (Open Web Text)

Max Activating Token Index: 903

Text #12

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3321. Min Act: -0.1700

Data Index: 7344639 (Open Web Text)

Max Activating Token Index: 329

Text #13

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3322. Min Act: -0.1700

Data Index: 5222558 (Open Web Text)

Max Activating Token Index: 323

Text #14

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3212. Min Act: -0.1700

Data Index: 1560747 (Open Web Text)

Max Activating Token Index: 475

Text #15

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3132. Min Act: -0.1700

Data Index: 7266340 (Open Web Text)

Max Activating Token Index: 972

Text #16

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3118. Min Act: -0.1700

Data Index: 7914920 (Open Web Text)

Max Activating Token Index: 991

Text #17

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3311. Min Act: -0.1700

Data Index: 2910284 (Open Web Text)

Max Activating Token Index: 240

Text #18

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3118. Min Act: -0.1700

Data Index: 703597 (Open Web Text)

Max Activating Token Index: 999

Text #19

Max Range: 3.5235. Min Range: -3.5235

Max Act: 3.3112. Min Act: -0.1700

Data Index: 2046345 (Open Web Text)

Max Activating Token Index: 842