Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1015 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.8691. Min Act: -0.1700

Data Index: 6745474 (Open Web Text)

Max Activating Token Index: 746

Text #1

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.8690. Min Act: -0.1700

Data Index: 2903648 (Open Web Text)

Max Activating Token Index: 16

Text #2

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.8854. Min Act: -0.1700

Data Index: 2347117 (Open Web Text)

Max Activating Token Index: 532

Text #3

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7895. Min Act: -0.1700

Data Index: 1933608 (Open Web Text)

Max Activating Token Index: 331

Text #4

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7671. Min Act: -0.1700

Data Index: 1687198 (Open Web Text)

Max Activating Token Index: 144

Text #5

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7447. Min Act: -0.1700

Data Index: 4390272 (Open Web Text)

Max Activating Token Index: 23

Text #6

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7380. Min Act: -0.1700

Data Index: 7416992 (Open Web Text)

Max Activating Token Index: 271

Text #7

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7342. Min Act: -0.1700

Data Index: 8618945 (Open Web Text)

Max Activating Token Index: 222

Text #8

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7350. Min Act: -0.1700

Data Index: 2753486 (Open Web Text)

Max Activating Token Index: 283

Text #9

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7454. Min Act: -0.1700

Data Index: 2293478 (Open Web Text)

Max Activating Token Index: 857

Text #10

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7487. Min Act: -0.1700

Data Index: 5776425 (Open Web Text)

Max Activating Token Index: 941

Text #11

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7268. Min Act: -0.1700

Data Index: 634552 (Open Web Text)

Max Activating Token Index: 388

Text #12

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7328. Min Act: -0.1700

Data Index: 2305841 (Open Web Text)

Max Activating Token Index: 125

Text #13

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7304. Min Act: -0.1700

Data Index: 5827465 (Open Web Text)

Max Activating Token Index: 60

Text #14

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7394. Min Act: -0.1700

Data Index: 8827355 (Open Web Text)

Max Activating Token Index: 192

Text #15

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7128. Min Act: -0.1700

Data Index: 4663215 (Open Web Text)

Max Activating Token Index: 969

Text #16

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.7072. Min Act: -0.1700

Data Index: 4823829 (Open Web Text)

Max Activating Token Index: 272

Text #17

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.6925. Min Act: -0.1700

Data Index: 4855573 (Open Web Text)

Max Activating Token Index: 923

Text #18

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.6921. Min Act: -0.1700

Data Index: 6552338 (Open Web Text)

Max Activating Token Index: 836

Text #19

Max Range: 3.8854. Min Range: -3.8854

Max Act: 3.6858. Min Act: -0.1700

Data Index: 7370495 (Open Web Text)

Max Activating Token Index: 901