Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2565 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9515. Min Act: -0.1700

Data Index: 3212527 (Open Web Text)

Max Activating Token Index: 785

Text #1

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9473. Min Act: -0.1700

Data Index: 1895902 (Open Web Text)

Max Activating Token Index: 646

Text #2

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9627. Min Act: -0.1700

Data Index: 8355596 (Open Web Text)

Max Activating Token Index: 920

Text #3

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9000. Min Act: -0.1700

Data Index: 4655112 (Open Web Text)

Max Activating Token Index: 821

Text #4

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9245. Min Act: -0.1700

Data Index: 2823822 (Open Web Text)

Max Activating Token Index: 914

Text #5

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9204. Min Act: -0.1700

Data Index: 8509736 (Open Web Text)

Max Activating Token Index: 373

Text #6

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.9095. Min Act: -0.1700

Data Index: 4503266 (Open Web Text)

Max Activating Token Index: 568

Text #7

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8854. Min Act: -0.1700

Data Index: 6693288 (Open Web Text)

Max Activating Token Index: 740

Text #8

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8628. Min Act: -0.1700

Data Index: 6169937 (Open Web Text)

Max Activating Token Index: 339

Text #9

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8550. Min Act: -0.1700

Data Index: 5822952 (Open Web Text)

Max Activating Token Index: 557

Text #10

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8720. Min Act: -0.1700

Data Index: 3404307 (Open Web Text)

Max Activating Token Index: 524

Text #11

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8628. Min Act: -0.1700

Data Index: 5065678 (Open Web Text)

Max Activating Token Index: 935

Text #12

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8358. Min Act: -0.1700

Data Index: 3691088 (Open Web Text)

Max Activating Token Index: 140

Text #13

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8359. Min Act: -0.1700

Data Index: 5744700 (Open Web Text)

Max Activating Token Index: 530

Text #14

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8185. Min Act: -0.1700

Data Index: 1334454 (Open Web Text)

Max Activating Token Index: 988

Text #15

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8299. Min Act: -0.1700

Data Index: 6815762 (Open Web Text)

Max Activating Token Index: 574

Text #16

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.7989. Min Act: -0.1700

Data Index: 286007 (Open Web Text)

Max Activating Token Index: 747

Text #17

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.8170. Min Act: -0.1700

Data Index: 4030110 (Open Web Text)

Max Activating Token Index: 322

Text #18

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.7776. Min Act: -0.1700

Data Index: 6457716 (Open Web Text)

Max Activating Token Index: 183

Text #19

Max Range: 3.9627. Min Range: -3.9627

Max Act: 3.7860. Min Act: -0.1700

Data Index: 4097930 (Open Web Text)

Max Activating Token Index: 230