Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 471 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6940. Min Act: -0.1700

Data Index: 5337573 (Open Web Text)

Max Activating Token Index: 522

Text #1

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6771. Min Act: -0.1700

Data Index: 2920528 (Open Web Text)

Max Activating Token Index: 433

Text #2

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6858. Min Act: -0.1700

Data Index: 394005 (Open Web Text)

Max Activating Token Index: 678

Text #3

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6696. Min Act: -0.1700

Data Index: 2420067 (Open Web Text)

Max Activating Token Index: 984

Text #4

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6457. Min Act: -0.1700

Data Index: 2964714 (Open Web Text)

Max Activating Token Index: 154

Text #5

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6653. Min Act: -0.1700

Data Index: 7587092 (Open Web Text)

Max Activating Token Index: 97

Text #6

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6590. Min Act: -0.1700

Data Index: 2593386 (Open Web Text)

Max Activating Token Index: 1019

Text #7

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6320. Min Act: -0.1700

Data Index: 7328403 (Open Web Text)

Max Activating Token Index: 23

Text #8

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6373. Min Act: -0.1700

Data Index: 6809508 (Open Web Text)

Max Activating Token Index: 389

Text #9

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6299. Min Act: -0.1700

Data Index: 6773821 (Open Web Text)

Max Activating Token Index: 74

Text #10

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6289. Min Act: -0.1700

Data Index: 5712492 (Open Web Text)

Max Activating Token Index: 1018

Text #11

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6325. Min Act: -0.1700

Data Index: 4723043 (Open Web Text)

Max Activating Token Index: 397

Text #12

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6255. Min Act: -0.1700

Data Index: 1909469 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6296. Min Act: -0.1700

Data Index: 4678189 (Open Web Text)

Max Activating Token Index: 554

Text #14

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6257. Min Act: -0.1700

Data Index: 5663644 (Open Web Text)

Max Activating Token Index: 718

Text #15

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6188. Min Act: -0.1700

Data Index: 1060207 (Open Web Text)

Max Activating Token Index: 273

Text #16

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6091. Min Act: -0.1700

Data Index: 2692629 (Open Web Text)

Max Activating Token Index: 873

Text #17

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6117. Min Act: -0.1698

Data Index: 944606 (Open Web Text)

Max Activating Token Index: 871

Text #18

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6236. Min Act: -0.1700

Data Index: 3841279 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 1.6940. Min Range: -1.6940

Max Act: 1.6182. Min Act: -0.1700

Data Index: 4125436 (Open Web Text)

Max Activating Token Index: 843