Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 19 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3683. Min Act: -0.1700

Data Index: 7897699 (Open Web Text)

Max Activating Token Index: 194

Text #1

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3275. Min Act: -0.1700

Data Index: 6245518 (Open Web Text)

Max Activating Token Index: 1007

Text #2

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3384. Min Act: -0.1700

Data Index: 93079 (Open Web Text)

Max Activating Token Index: 252

Text #3

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3185. Min Act: -0.1700

Data Index: 518949 (Open Web Text)

Max Activating Token Index: 365

Text #4

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3108. Min Act: -0.1700

Data Index: 518956 (Open Web Text)

Max Activating Token Index: 898

Text #5

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3062. Min Act: -0.1700

Data Index: 93078 (Open Web Text)

Max Activating Token Index: 120

Text #6

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3046. Min Act: -0.1700

Data Index: 297222 (Open Web Text)

Max Activating Token Index: 348

Text #7

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.3096. Min Act: -0.1700

Data Index: 7508389 (Open Web Text)

Max Activating Token Index: 431

Text #8

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2873. Min Act: -0.1700

Data Index: 6709412 (Open Web Text)

Max Activating Token Index: 363

Text #9

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2805. Min Act: -0.1700

Data Index: 8626018 (Open Web Text)

Max Activating Token Index: 448

Text #10

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2562. Min Act: -0.1700

Data Index: 8105600 (Open Web Text)

Max Activating Token Index: 551

Text #11

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2644. Min Act: -0.1700

Data Index: 3583978 (Open Web Text)

Max Activating Token Index: 652

Text #12

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2736. Min Act: -0.1700

Data Index: 7739554 (Open Web Text)

Max Activating Token Index: 860

Text #13

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2717. Min Act: -0.1700

Data Index: 6267373 (Open Web Text)

Max Activating Token Index: 751

Text #14

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2650. Min Act: -0.1700

Data Index: 6467619 (Open Web Text)

Max Activating Token Index: 768

Text #15

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2427. Min Act: -0.1700

Data Index: 8060055 (Open Web Text)

Max Activating Token Index: 1007

Text #16

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2417. Min Act: -0.1700

Data Index: 1197338 (Open Web Text)

Max Activating Token Index: 813

Text #17

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2449. Min Act: -0.1700

Data Index: 5424552 (Open Web Text)

Max Activating Token Index: 474

Text #18

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2489. Min Act: -0.1700

Data Index: 3697716 (Open Web Text)

Max Activating Token Index: 326

Text #19

Max Range: 3.3683. Min Range: -3.3683

Max Act: 3.2590. Min Act: -0.1700

Data Index: 5527601 (Open Web Text)

Max Activating Token Index: 873