Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1880 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.7269. Min Act: -0.1700

Data Index: 1761266 (Open Web Text)

Max Activating Token Index: 858

Text #1

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.7167. Min Act: -0.1698

Data Index: 4750278 (Open Web Text)

Max Activating Token Index: 761

Text #2

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6827. Min Act: -0.1699

Data Index: 3681465 (Open Web Text)

Max Activating Token Index: 542

Text #3

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6638. Min Act: -0.1686

Data Index: 6757935 (Open Web Text)

Max Activating Token Index: 582

Text #4

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6614. Min Act: -0.1698

Data Index: 7506238 (Open Web Text)

Max Activating Token Index: 138

Text #5

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6581. Min Act: -0.1700

Data Index: 5301866 (Open Web Text)

Max Activating Token Index: 970

Text #6

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6611. Min Act: -0.1700

Data Index: 8575882 (Open Web Text)

Max Activating Token Index: 785

Text #7

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6609. Min Act: -0.1700

Data Index: 463771 (Open Web Text)

Max Activating Token Index: 536

Text #8

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6495. Min Act: -0.1688

Data Index: 7369936 (Open Web Text)

Max Activating Token Index: 539

Text #9

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6407. Min Act: -0.1692

Data Index: 3966046 (Open Web Text)

Max Activating Token Index: 558

Text #10

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6343. Min Act: -0.1699

Data Index: 243565 (Open Web Text)

Max Activating Token Index: 310

Text #11

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6364. Min Act: -0.1700

Data Index: 6626465 (Open Web Text)

Max Activating Token Index: 188

Text #12

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6383. Min Act: -0.1700

Data Index: 5976092 (Open Web Text)

Max Activating Token Index: 303

Text #13

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6390. Min Act: -0.1700

Data Index: 4991560 (Open Web Text)

Max Activating Token Index: 332

Text #14

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6355. Min Act: -0.1700

Data Index: 336460 (Open Web Text)

Max Activating Token Index: 381

Text #15

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6414. Min Act: -0.1692

Data Index: 8348350 (Open Web Text)

Max Activating Token Index: 224

Text #16

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6348. Min Act: -0.1696

Data Index: 1010699 (Open Web Text)

Max Activating Token Index: 708

Text #17

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6278. Min Act: -0.1700

Data Index: 1797120 (Open Web Text)

Max Activating Token Index: 505

Text #18

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6303. Min Act: -0.1699

Data Index: 5135749 (Open Web Text)

Max Activating Token Index: 629

Text #19

Max Range: 1.7269. Min Range: -1.7269

Max Act: 1.6268. Min Act: -0.1699

Data Index: 7050973 (Open Web Text)

Max Activating Token Index: 306