Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 620 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.8156. Min Act: -0.1700

Data Index: 7845263 (Open Web Text)

Max Activating Token Index: 17

Text #1

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.7156. Min Act: -0.1700

Data Index: 3389853 (Open Web Text)

Max Activating Token Index: 12

Text #2

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.7286. Min Act: -0.1700

Data Index: 2025553 (Open Web Text)

Max Activating Token Index: 15

Text #3

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.6770. Min Act: -0.1700

Data Index: 259580 (Open Web Text)

Max Activating Token Index: 20

Text #4

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.5577. Min Act: -0.1700

Data Index: 611946 (Open Web Text)

Max Activating Token Index: 32

Text #5

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.5574. Min Act: -0.1700

Data Index: 8445989 (Open Web Text)

Max Activating Token Index: 28

Text #6

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.4569. Min Act: -0.1700

Data Index: 2523275 (Open Web Text)

Max Activating Token Index: 205

Text #7

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.4389. Min Act: -0.1700

Data Index: 5358231 (Open Web Text)

Max Activating Token Index: 614

Text #8

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.4834. Min Act: -0.1700

Data Index: 2117981 (Open Web Text)

Max Activating Token Index: 636

Text #9

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.4786. Min Act: -0.1700

Data Index: 5827344 (Open Web Text)

Max Activating Token Index: 25

Text #10

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3642. Min Act: -0.1700

Data Index: 2540265 (Open Web Text)

Max Activating Token Index: 574

Text #11

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3725. Min Act: -0.1700

Data Index: 7946252 (Open Web Text)

Max Activating Token Index: 1011

Text #12

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3362. Min Act: -0.1700

Data Index: 873421 (Open Web Text)

Max Activating Token Index: 416

Text #13

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3636. Min Act: -0.1700

Data Index: 5219915 (Open Web Text)

Max Activating Token Index: 45

Text #14

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3165. Min Act: -0.1700

Data Index: 3482153 (Open Web Text)

Max Activating Token Index: 20

Text #15

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3420. Min Act: -0.1700

Data Index: 3735922 (Open Web Text)

Max Activating Token Index: 21

Text #16

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3269. Min Act: -0.1700

Data Index: 5271574 (Open Web Text)

Max Activating Token Index: 142

Text #17

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.2798. Min Act: -0.1700

Data Index: 2868226 (Open Web Text)

Max Activating Token Index: 268

Text #18

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3390. Min Act: -0.1700

Data Index: 4139146 (Open Web Text)

Max Activating Token Index: 31

Text #19

Max Range: 4.8156. Min Range: -4.8156

Max Act: 4.3141. Min Act: -0.1700

Data Index: 1240473 (Open Web Text)

Max Activating Token Index: 5