Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1009 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.9445. Min Act: -0.1700

Data Index: 1614634 (Open Web Text)

Max Activating Token Index: 911

Text #1

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8914. Min Act: -0.1700

Data Index: 4196682 (Open Web Text)

Max Activating Token Index: 818

Text #2

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8951. Min Act: -0.1700

Data Index: 7874772 (Open Web Text)

Max Activating Token Index: 884

Text #3

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.9049. Min Act: -0.1700

Data Index: 7221295 (Open Web Text)

Max Activating Token Index: 970

Text #4

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8927. Min Act: -0.1700

Data Index: 2362996 (Open Web Text)

Max Activating Token Index: 821

Text #5

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8710. Min Act: -0.1700

Data Index: 5841167 (Open Web Text)

Max Activating Token Index: 941

Text #6

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8483. Min Act: -0.1700

Data Index: 3282721 (Open Web Text)

Max Activating Token Index: 418

Text #7

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8799. Min Act: -0.1700

Data Index: 6632188 (Open Web Text)

Max Activating Token Index: 912

Text #8

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8461. Min Act: -0.1700

Data Index: 7574893 (Open Web Text)

Max Activating Token Index: 890

Text #9

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8212. Min Act: -0.1700

Data Index: 5526445 (Open Web Text)

Max Activating Token Index: 949

Text #10

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8305. Min Act: -0.1700

Data Index: 602894 (Open Web Text)

Max Activating Token Index: 980

Text #11

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8200. Min Act: -0.1700

Data Index: 8739855 (Open Web Text)

Max Activating Token Index: 697

Text #12

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8135. Min Act: -0.1700

Data Index: 784322 (Open Web Text)

Max Activating Token Index: 907

Text #13

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8165. Min Act: -0.1700

Data Index: 6137441 (Open Web Text)

Max Activating Token Index: 432

Text #14

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8304. Min Act: -0.1700

Data Index: 5697061 (Open Web Text)

Max Activating Token Index: 821

Text #15

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8283. Min Act: -0.1700

Data Index: 2000697 (Open Web Text)

Max Activating Token Index: 943

Text #16

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8473. Min Act: -0.1700

Data Index: 2467840 (Open Web Text)

Max Activating Token Index: 453

Text #17

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8373. Min Act: -0.1700

Data Index: 1013382 (Open Web Text)

Max Activating Token Index: 831

Text #18

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.8097. Min Act: -0.1700

Data Index: 588871 (Open Web Text)

Max Activating Token Index: 690

Text #19

Max Range: 3.9445. Min Range: -3.9445

Max Act: 3.7947. Min Act: -0.1700

Data Index: 4879233 (Open Web Text)

Max Activating Token Index: 908