Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2598 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.8869. Min Act: -0.1700

Data Index: 2398833 (Open Web Text)

Max Activating Token Index: 832

Text #1

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.8609. Min Act: -0.1700

Data Index: 4271336 (Open Web Text)

Max Activating Token Index: 848

Text #2

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.8409. Min Act: -0.1700

Data Index: 6552574 (Open Web Text)

Max Activating Token Index: 42

Text #3

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.8072. Min Act: -0.1700

Data Index: 6886740 (Open Web Text)

Max Activating Token Index: 501

Text #4

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7987. Min Act: -0.1700

Data Index: 5257094 (Open Web Text)

Max Activating Token Index: 369

Text #5

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7460. Min Act: -0.1700

Data Index: 261776 (Open Web Text)

Max Activating Token Index: 853

Text #6

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7698. Min Act: -0.1700

Data Index: 5272148 (Open Web Text)

Max Activating Token Index: 79

Text #7

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7704. Min Act: -0.1700

Data Index: 1549688 (Open Web Text)

Max Activating Token Index: 376

Text #8

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7666. Min Act: -0.1700

Data Index: 2390234 (Open Web Text)

Max Activating Token Index: 337

Text #9

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7463. Min Act: -0.1700

Data Index: 4987585 (Open Web Text)

Max Activating Token Index: 428

Text #10

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.7547. Min Act: -0.1700

Data Index: 2831821 (Open Web Text)

Max Activating Token Index: 684

Text #11

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6931. Min Act: -0.1700

Data Index: 8344324 (Open Web Text)

Max Activating Token Index: 344

Text #12

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6728. Min Act: -0.1700

Data Index: 6785832 (Open Web Text)

Max Activating Token Index: 758

Text #13

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6627. Min Act: -0.1700

Data Index: 5432396 (Open Web Text)

Max Activating Token Index: 194

Text #14

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6345. Min Act: -0.1700

Data Index: 134164 (Open Web Text)

Max Activating Token Index: 226

Text #15

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6511. Min Act: -0.1700

Data Index: 8124556 (Open Web Text)

Max Activating Token Index: 764

Text #16

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6400. Min Act: -0.1700

Data Index: 666261 (Open Web Text)

Max Activating Token Index: 719

Text #17

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.6213. Min Act: -0.1700

Data Index: 4573841 (Open Web Text)

Max Activating Token Index: 941

Text #18

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.5428. Min Act: -0.1700

Data Index: 6499548 (Open Web Text)

Max Activating Token Index: 422

Text #19

Max Range: 3.8869. Min Range: -3.8869

Max Act: 3.5710. Min Act: -0.1700

Data Index: 3426989 (Open Web Text)

Max Activating Token Index: 609