Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2554 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0227. Min Range: -3.0227

Max Act: 3.0227. Min Act: -0.1700

Data Index: 5159511 (Open Web Text)

Max Activating Token Index: 7

Text #1

Max Range: 3.0227. Min Range: -3.0227

Max Act: 3.0158. Min Act: -0.1700

Data Index: 47107 (Open Web Text)

Max Activating Token Index: 72

Text #2

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9570. Min Act: -0.1700

Data Index: 437608 (Open Web Text)

Max Activating Token Index: 7

Text #3

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9370. Min Act: -0.1700

Data Index: 7090782 (Open Web Text)

Max Activating Token Index: 13

Text #4

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9236. Min Act: -0.1700

Data Index: 5961688 (Open Web Text)

Max Activating Token Index: 95

Text #5

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9257. Min Act: -0.1700

Data Index: 5661798 (Open Web Text)

Max Activating Token Index: 852

Text #6

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9045. Min Act: -0.1700

Data Index: 1496890 (Open Web Text)

Max Activating Token Index: 448

Text #7

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.9021. Min Act: -0.1700

Data Index: 6076853 (Open Web Text)

Max Activating Token Index: 512

Text #8

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8341. Min Act: -0.1700

Data Index: 2803223 (Open Web Text)

Max Activating Token Index: 572

Text #9

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8562. Min Act: -0.1700

Data Index: 2512943 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8350. Min Act: -0.1700

Data Index: 5676149 (Open Web Text)

Max Activating Token Index: 451

Text #11

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8242. Min Act: -0.1700

Data Index: 4027192 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8207. Min Act: -0.1700

Data Index: 4533488 (Open Web Text)

Max Activating Token Index: 907

Text #13

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.8156. Min Act: -0.1700

Data Index: 5654217 (Open Web Text)

Max Activating Token Index: 435

Text #14

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7966. Min Act: -0.1700

Data Index: 4533490 (Open Web Text)

Max Activating Token Index: 436

Text #15

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7955. Min Act: -0.1700

Data Index: 2618061 (Open Web Text)

Max Activating Token Index: 17

Text #16

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7915. Min Act: -0.1700

Data Index: 5848766 (Open Web Text)

Max Activating Token Index: 10

Text #17

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7909. Min Act: -0.1700

Data Index: 3849835 (Open Web Text)

Max Activating Token Index: 120

Text #18

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7999. Min Act: -0.1700

Data Index: 3970131 (Open Web Text)

Max Activating Token Index: 353

Text #19

Max Range: 3.0227. Min Range: -3.0227

Max Act: 2.7925. Min Act: -0.1700

Data Index: 7075933 (Open Web Text)

Max Activating Token Index: 126