Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2544 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.5164. Min Act: -0.1700

Data Index: 5038997 (Open Web Text)

Max Activating Token Index: 200

Text #1

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.5240. Min Act: -0.1699

Data Index: 6880081 (Open Web Text)

Max Activating Token Index: 179

Text #2

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.5206. Min Act: -0.1700

Data Index: 7458203 (Open Web Text)

Max Activating Token Index: 181

Text #3

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.5135. Min Act: -0.1700

Data Index: 162727 (Open Web Text)

Max Activating Token Index: 753

Text #4

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4922. Min Act: -0.1700

Data Index: 8680326 (Open Web Text)

Max Activating Token Index: 342

Text #5

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4876. Min Act: -0.1700

Data Index: 6060160 (Open Web Text)

Max Activating Token Index: 210

Text #6

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4915. Min Act: -0.1700

Data Index: 6581431 (Open Web Text)

Max Activating Token Index: 105

Text #7

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4902. Min Act: -0.1700

Data Index: 6606042 (Open Web Text)

Max Activating Token Index: 422

Text #8

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4810. Min Act: -0.1677

Data Index: 403640 (Open Web Text)

Max Activating Token Index: 436

Text #9

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4813. Min Act: -0.1700

Data Index: 6269423 (Open Web Text)

Max Activating Token Index: 743

Text #10

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4832. Min Act: -0.1700

Data Index: 7291313 (Open Web Text)

Max Activating Token Index: 343

Text #11

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4759. Min Act: -0.1700

Data Index: 239881 (Open Web Text)

Max Activating Token Index: 310

Text #12

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4585. Min Act: -0.1700

Data Index: 6769290 (Open Web Text)

Max Activating Token Index: 341

Text #13

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4623. Min Act: -0.1700

Data Index: 5700476 (Open Web Text)

Max Activating Token Index: 106

Text #14

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4655. Min Act: -0.1700

Data Index: 1746920 (Open Web Text)

Max Activating Token Index: 456

Text #15

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4628. Min Act: -0.1700

Data Index: 3048523 (Open Web Text)

Max Activating Token Index: 675

Text #16

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4643. Min Act: -0.1700

Data Index: 8468900 (Open Web Text)

Max Activating Token Index: 814

Text #17

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4504. Min Act: -0.1700

Data Index: 6525593 (Open Web Text)

Max Activating Token Index: 233

Text #18

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4493. Min Act: -0.1700

Data Index: 4420643 (Open Web Text)

Max Activating Token Index: 562

Text #19

Max Range: 1.5240. Min Range: -1.5240

Max Act: 1.4486. Min Act: -0.1700

Data Index: 2017816 (Open Web Text)

Max Activating Token Index: 412