Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1498 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3337. Min Range: -4.3337

Max Act: 4.3337. Min Act: -0.1700

Data Index: 6597089 (Open Web Text)

Max Activating Token Index: 425

Text #1

Max Range: 4.3337. Min Range: -4.3337

Max Act: 4.1007. Min Act: -0.1700

Data Index: 2052271 (Open Web Text)

Max Activating Token Index: 550

Text #2

Max Range: 4.3337. Min Range: -4.3337

Max Act: 4.0447. Min Act: -0.1700

Data Index: 2392192 (Open Web Text)

Max Activating Token Index: 484

Text #3

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8632. Min Act: -0.1700

Data Index: 5042556 (Open Web Text)

Max Activating Token Index: 27

Text #4

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8877. Min Act: -0.1489

Data Index: 4137936 (Open Web Text)

Max Activating Token Index: 974

Text #5

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8807. Min Act: -0.1700

Data Index: 6690517 (Open Web Text)

Max Activating Token Index: 905

Text #6

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8734. Min Act: -0.1699

Data Index: 6448115 (Open Web Text)

Max Activating Token Index: 387

Text #7

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8880. Min Act: -0.1700

Data Index: 8162052 (Open Web Text)

Max Activating Token Index: 132

Text #8

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8386. Min Act: -0.1700

Data Index: 491351 (Open Web Text)

Max Activating Token Index: 348

Text #9

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8401. Min Act: -0.1700

Data Index: 5615237 (Open Web Text)

Max Activating Token Index: 674

Text #10

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8531. Min Act: -0.1700

Data Index: 4497152 (Open Web Text)

Max Activating Token Index: 759

Text #11

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8517. Min Act: -0.1683

Data Index: 1041869 (Open Web Text)

Max Activating Token Index: 931

Text #12

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.8298. Min Act: -0.1692

Data Index: 1904640 (Open Web Text)

Max Activating Token Index: 176

Text #13

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7857. Min Act: -0.1700

Data Index: 3683863 (Open Web Text)

Max Activating Token Index: 916

Text #14

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7926. Min Act: -0.1700

Data Index: 734163 (Open Web Text)

Max Activating Token Index: 976

Text #15

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7868. Min Act: -0.1699

Data Index: 1320161 (Open Web Text)

Max Activating Token Index: 20

Text #16

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7968. Min Act: -0.1700

Data Index: 4474439 (Open Web Text)

Max Activating Token Index: 590

Text #17

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7800. Min Act: -0.1700

Data Index: 4059205 (Open Web Text)

Max Activating Token Index: 300

Text #18

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7628. Min Act: -0.1645

Data Index: 5747517 (Open Web Text)

Max Activating Token Index: 1020

Text #19

Max Range: 4.3337. Min Range: -4.3337

Max Act: 3.7572. Min Act: -0.1700

Data Index: 1745648 (Open Web Text)

Max Activating Token Index: 49