Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1993 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0009. Min Range: -4.0009

Max Act: 4.0009. Min Act: -0.1700

Data Index: 1762401 (Open Web Text)

Max Activating Token Index: 112

Text #1

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.9706. Min Act: -0.1700

Data Index: 8130054 (Open Web Text)

Max Activating Token Index: 336

Text #2

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.9522. Min Act: -0.1700

Data Index: 2413244 (Open Web Text)

Max Activating Token Index: 271

Text #3

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8876. Min Act: -0.1700

Data Index: 2771180 (Open Web Text)

Max Activating Token Index: 504

Text #4

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.9200. Min Act: -0.1700

Data Index: 7742001 (Open Web Text)

Max Activating Token Index: 470

Text #5

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8922. Min Act: -0.1700

Data Index: 7950174 (Open Web Text)

Max Activating Token Index: 336

Text #6

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8434. Min Act: -0.1700

Data Index: 583477 (Open Web Text)

Max Activating Token Index: 378

Text #7

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8541. Min Act: -0.1700

Data Index: 5057526 (Open Web Text)

Max Activating Token Index: 1000

Text #8

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8463. Min Act: -0.1700

Data Index: 4651854 (Open Web Text)

Max Activating Token Index: 41

Text #9

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8541. Min Act: -0.1700

Data Index: 2297660 (Open Web Text)

Max Activating Token Index: 145

Text #10

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8472. Min Act: -0.1700

Data Index: 3414483 (Open Web Text)

Max Activating Token Index: 269

Text #11

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8432. Min Act: -0.1700

Data Index: 7732688 (Open Web Text)

Max Activating Token Index: 496

Text #12

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8515. Min Act: -0.1700

Data Index: 2104588 (Open Web Text)

Max Activating Token Index: 15

Text #13

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8699. Min Act: -0.1700

Data Index: 3518327 (Open Web Text)

Max Activating Token Index: 187

Text #14

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8438. Min Act: -0.1700

Data Index: 8276004 (Open Web Text)

Max Activating Token Index: 104

Text #15

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.7941. Min Act: -0.1700

Data Index: 1509780 (Open Web Text)

Max Activating Token Index: 89

Text #16

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8216. Min Act: -0.1700

Data Index: 5812297 (Open Web Text)

Max Activating Token Index: 236

Text #17

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8390. Min Act: -0.1700

Data Index: 2653295 (Open Web Text)

Max Activating Token Index: 941

Text #18

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8224. Min Act: -0.1700

Data Index: 7076322 (Open Web Text)

Max Activating Token Index: 352

Text #19

Max Range: 4.0009. Min Range: -4.0009

Max Act: 3.8021. Min Act: -0.1700

Data Index: 5345044 (Open Web Text)

Max Activating Token Index: 56