Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1818 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.5117. Min Act: -0.1700

Data Index: 427380 (Open Web Text)

Max Activating Token Index: 373

Text #1

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.4897. Min Act: -0.1700

Data Index: 2829534 (Open Web Text)

Max Activating Token Index: 391

Text #2

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.4167. Min Act: -0.1700

Data Index: 3302646 (Open Web Text)

Max Activating Token Index: 202

Text #3

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.4161. Min Act: -0.1700

Data Index: 7607221 (Open Web Text)

Max Activating Token Index: 910

Text #4

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.4303. Min Act: -0.1700

Data Index: 3234612 (Open Web Text)

Max Activating Token Index: 155

Text #5

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.3456. Min Act: -0.1700

Data Index: 3562214 (Open Web Text)

Max Activating Token Index: 636

Text #6

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.3265. Min Act: -0.1700

Data Index: 3867966 (Open Web Text)

Max Activating Token Index: 249

Text #7

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.3417. Min Act: -0.1700

Data Index: 5628126 (Open Web Text)

Max Activating Token Index: 115

Text #8

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.3333. Min Act: -0.1700

Data Index: 1211811 (Open Web Text)

Max Activating Token Index: 222

Text #9

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2887. Min Act: -0.1700

Data Index: 5439056 (Open Web Text)

Max Activating Token Index: 857

Text #10

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2899. Min Act: -0.1700

Data Index: 8076916 (Open Web Text)

Max Activating Token Index: 372

Text #11

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2831. Min Act: -0.1700

Data Index: 3577814 (Open Web Text)

Max Activating Token Index: 653

Text #12

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.3029. Min Act: -0.1700

Data Index: 7923187 (Open Web Text)

Max Activating Token Index: 197

Text #13

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2826. Min Act: -0.1700

Data Index: 1485980 (Open Web Text)

Max Activating Token Index: 374

Text #14

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2831. Min Act: -0.1700

Data Index: 1604273 (Open Web Text)

Max Activating Token Index: 529

Text #15

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2451. Min Act: -0.1700

Data Index: 21174 (Open Web Text)

Max Activating Token Index: 541

Text #16

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2369. Min Act: -0.1700

Data Index: 3863331 (Open Web Text)

Max Activating Token Index: 531

Text #17

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2810. Min Act: -0.1700

Data Index: 4134171 (Open Web Text)

Max Activating Token Index: 286

Text #18

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2491. Min Act: -0.1700

Data Index: 1756070 (Open Web Text)

Max Activating Token Index: 146

Text #19

Max Range: 3.5117. Min Range: -3.5117

Max Act: 3.2437. Min Act: -0.1700

Data Index: 4671858 (Open Web Text)

Max Activating Token Index: 990