Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2657 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.5346. Min Act: -0.1700

Data Index: 8774270 (Open Web Text)

Max Activating Token Index: 181

Text #1

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.5241. Min Act: -0.1700

Data Index: 3691438 (Open Web Text)

Max Activating Token Index: 23

Text #2

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.5370. Min Act: -0.1700

Data Index: 3962007 (Open Web Text)

Max Activating Token Index: 109

Text #3

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.5061. Min Act: -0.1700

Data Index: 4798208 (Open Web Text)

Max Activating Token Index: 685

Text #4

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4586. Min Act: -0.1700

Data Index: 7340054 (Open Web Text)

Max Activating Token Index: 907

Text #5

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4857. Min Act: -0.1700

Data Index: 5869314 (Open Web Text)

Max Activating Token Index: 812

Text #6

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4622. Min Act: -0.1700

Data Index: 4962317 (Open Web Text)

Max Activating Token Index: 204

Text #7

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4560. Min Act: -0.1700

Data Index: 724502 (Open Web Text)

Max Activating Token Index: 43

Text #8

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4775. Min Act: -0.1700

Data Index: 8398138 (Open Web Text)

Max Activating Token Index: 327

Text #9

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4449. Min Act: -0.1700

Data Index: 2384529 (Open Web Text)

Max Activating Token Index: 822

Text #10

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4335. Min Act: -0.1700

Data Index: 7227907 (Open Web Text)

Max Activating Token Index: 808

Text #11

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4519. Min Act: -0.1700

Data Index: 1143487 (Open Web Text)

Max Activating Token Index: 575

Text #12

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4374. Min Act: -0.1700

Data Index: 6290469 (Open Web Text)

Max Activating Token Index: 426

Text #13

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4186. Min Act: -0.1700

Data Index: 6056028 (Open Web Text)

Max Activating Token Index: 1021

Text #14

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4201. Min Act: -0.1700

Data Index: 7926420 (Open Web Text)

Max Activating Token Index: 682

Text #15

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4051. Min Act: -0.1700

Data Index: 3778219 (Open Web Text)

Max Activating Token Index: 954

Text #16

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4367. Min Act: -0.1700

Data Index: 7524533 (Open Web Text)

Max Activating Token Index: 606

Text #17

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4193. Min Act: -0.1700

Data Index: 61385 (Open Web Text)

Max Activating Token Index: 7

Text #18

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.4490. Min Act: -0.1700

Data Index: 5262660 (Open Web Text)

Max Activating Token Index: 369

Text #19

Max Range: 3.5370. Min Range: -3.5370

Max Act: 3.3971. Min Act: -0.1700

Data Index: 4556814 (Open Web Text)

Max Activating Token Index: 410