Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1813 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.5347. Min Act: -0.1700

Data Index: 7611743 (Open Web Text)

Max Activating Token Index: 14

Text #1

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.4180. Min Act: -0.1700

Data Index: 4831318 (Open Web Text)

Max Activating Token Index: 98

Text #2

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.3882. Min Act: -0.1700

Data Index: 2184510 (Open Web Text)

Max Activating Token Index: 884

Text #3

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2560. Min Act: -0.1700

Data Index: 5383159 (Open Web Text)

Max Activating Token Index: 141

Text #4

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2859. Min Act: -0.1700

Data Index: 8456020 (Open Web Text)

Max Activating Token Index: 416

Text #5

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2884. Min Act: -0.1700

Data Index: 1535891 (Open Web Text)

Max Activating Token Index: 20

Text #6

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.3199. Min Act: -0.1700

Data Index: 3514883 (Open Web Text)

Max Activating Token Index: 581

Text #7

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2580. Min Act: -0.1700

Data Index: 5958542 (Open Web Text)

Max Activating Token Index: 107

Text #8

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2397. Min Act: -0.1700

Data Index: 2751338 (Open Web Text)

Max Activating Token Index: 334

Text #9

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2563. Min Act: -0.1700

Data Index: 2198409 (Open Web Text)

Max Activating Token Index: 772

Text #10

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.2185. Min Act: -0.1700

Data Index: 2799864 (Open Web Text)

Max Activating Token Index: 70

Text #11

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1821. Min Act: -0.1700

Data Index: 6657056 (Open Web Text)

Max Activating Token Index: 72

Text #12

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1905. Min Act: -0.1700

Data Index: 5116735 (Open Web Text)

Max Activating Token Index: 1007

Text #13

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1898. Min Act: -0.1700

Data Index: 662015 (Open Web Text)

Max Activating Token Index: 556

Text #14

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1755. Min Act: -0.1700

Data Index: 8625685 (Open Web Text)

Max Activating Token Index: 320

Text #15

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1328. Min Act: -0.1700

Data Index: 2957844 (Open Web Text)

Max Activating Token Index: 830

Text #16

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1774. Min Act: -0.1700

Data Index: 7073830 (Open Web Text)

Max Activating Token Index: 528

Text #17

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1734. Min Act: -0.1700

Data Index: 5411013 (Open Web Text)

Max Activating Token Index: 414

Text #18

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1374. Min Act: -0.1700

Data Index: 7115729 (Open Web Text)

Max Activating Token Index: 319

Text #19

Max Range: 5.5347. Min Range: -5.5347

Max Act: 5.1363. Min Act: -0.1700

Data Index: 221098 (Open Web Text)

Max Activating Token Index: 59