Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 999 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0835. Min Act: -0.1700

Data Index: 924413 (Open Web Text)

Max Activating Token Index: 213

Text #1

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0633. Min Act: -0.1700

Data Index: 4548832 (Open Web Text)

Max Activating Token Index: 146

Text #2

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0519. Min Act: -0.1700

Data Index: 7529287 (Open Web Text)

Max Activating Token Index: 257

Text #3

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0387. Min Act: -0.1700

Data Index: 1148502 (Open Web Text)

Max Activating Token Index: 998

Text #4

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0216. Min Act: -0.1700

Data Index: 3766973 (Open Web Text)

Max Activating Token Index: 856

Text #5

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0260. Min Act: -0.1700

Data Index: 1475987 (Open Web Text)

Max Activating Token Index: 600

Text #6

Max Range: 3.0835. Min Range: -3.0835

Max Act: 3.0076. Min Act: -0.1700

Data Index: 6540206 (Open Web Text)

Max Activating Token Index: 229

Text #7

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9929. Min Act: -0.1700

Data Index: 7132665 (Open Web Text)

Max Activating Token Index: 1021

Text #8

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9572. Min Act: -0.1700

Data Index: 1797096 (Open Web Text)

Max Activating Token Index: 344

Text #9

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9664. Min Act: -0.1700

Data Index: 6717111 (Open Web Text)

Max Activating Token Index: 131

Text #10

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9772. Min Act: -0.1700

Data Index: 3201448 (Open Web Text)

Max Activating Token Index: 751

Text #11

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9303. Min Act: -0.1700

Data Index: 1471833 (Open Web Text)

Max Activating Token Index: 943

Text #12

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9415. Min Act: -0.1700

Data Index: 5315019 (Open Web Text)

Max Activating Token Index: 536

Text #13

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9540. Min Act: -0.1700

Data Index: 2775579 (Open Web Text)

Max Activating Token Index: 62

Text #14

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9560. Min Act: -0.1700

Data Index: 8446331 (Open Web Text)

Max Activating Token Index: 755

Text #15

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9440. Min Act: -0.1700

Data Index: 2627247 (Open Web Text)

Max Activating Token Index: 1003

Text #16

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9541. Min Act: -0.1700

Data Index: 170534 (Open Web Text)

Max Activating Token Index: 233

Text #17

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9091. Min Act: -0.1700

Data Index: 1942509 (Open Web Text)

Max Activating Token Index: 111

Text #18

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9225. Min Act: -0.1700

Data Index: 231695 (Open Web Text)

Max Activating Token Index: 372

Text #19

Max Range: 3.0835. Min Range: -3.0835

Max Act: 2.9126. Min Act: -0.1700

Data Index: 411533 (Open Web Text)

Max Activating Token Index: 85