Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2703 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.8360. Min Act: -0.1700

Data Index: 6802671 (Open Web Text)

Max Activating Token Index: 154

Text #1

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.7231. Min Act: -0.1700

Data Index: 4113875 (Open Web Text)

Max Activating Token Index: 431

Text #2

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.7151. Min Act: -0.1700

Data Index: 7053500 (Open Web Text)

Max Activating Token Index: 286

Text #3

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6950. Min Act: -0.1700

Data Index: 5376543 (Open Web Text)

Max Activating Token Index: 240

Text #4

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6985. Min Act: -0.1700

Data Index: 1973962 (Open Web Text)

Max Activating Token Index: 844

Text #5

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6657. Min Act: -0.1700

Data Index: 8190684 (Open Web Text)

Max Activating Token Index: 165

Text #6

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6414. Min Act: -0.1700

Data Index: 3794752 (Open Web Text)

Max Activating Token Index: 326

Text #7

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6869. Min Act: -0.1700

Data Index: 3774947 (Open Web Text)

Max Activating Token Index: 343

Text #8

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6525. Min Act: -0.1700

Data Index: 322022 (Open Web Text)

Max Activating Token Index: 624

Text #9

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6544. Min Act: -0.1700

Data Index: 3193463 (Open Web Text)

Max Activating Token Index: 277

Text #10

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6815. Min Act: -0.1700

Data Index: 6779204 (Open Web Text)

Max Activating Token Index: 87

Text #11

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6955. Min Act: -0.1697

Data Index: 34234 (Open Web Text)

Max Activating Token Index: 538

Text #12

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6792. Min Act: -0.1700

Data Index: 8237097 (Open Web Text)

Max Activating Token Index: 214

Text #13

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6297. Min Act: -0.1698

Data Index: 2645744 (Open Web Text)

Max Activating Token Index: 207

Text #14

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6475. Min Act: -0.1700

Data Index: 1890365 (Open Web Text)

Max Activating Token Index: 719

Text #15

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6258. Min Act: -0.1700

Data Index: 3073457 (Open Web Text)

Max Activating Token Index: 420

Text #16

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6448. Min Act: -0.1700

Data Index: 6875855 (Open Web Text)

Max Activating Token Index: 783

Text #17

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6563. Min Act: -0.1700

Data Index: 3366473 (Open Web Text)

Max Activating Token Index: 478

Text #18

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6374. Min Act: -0.1700

Data Index: 3573843 (Open Web Text)

Max Activating Token Index: 144

Text #19

Max Range: 5.8360. Min Range: -5.8360

Max Act: 5.6236. Min Act: -0.1700

Data Index: 6667617 (Open Web Text)

Max Activating Token Index: 1008