Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1005 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.2254. Min Act: -0.0992

Data Index: 1003045 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.2095. Min Act: -0.0556

Data Index: 7545925 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1952. Min Act: -0.0356

Data Index: 2185949 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1707. Min Act: -0.0795

Data Index: 8359798 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1742. Min Act: -0.0611

Data Index: 6353769 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1599. Min Act: -0.0249

Data Index: 8578003 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1510. Min Act: -0.0596

Data Index: 4768558 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1585. Min Act: -0.0753

Data Index: 2782132 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1578. Min Act: -0.0611

Data Index: 7125160 (Open Web Text)

Max Activating Token Index: 913

Text #9

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1548. Min Act: -0.0085

Data Index: 3016702 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1569. Min Act: -0.0937

Data Index: 1197614 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1447. Min Act: -0.0333

Data Index: 1343257 (Open Web Text)

Max Activating Token Index: 912

Text #12

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1447. Min Act: -0.0842

Data Index: 7877663 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1460. Min Act: -0.0860

Data Index: 2945419 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1502. Min Act: -0.0486

Data Index: 860341 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1342. Min Act: -0.0854

Data Index: 5793597 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1414. Min Act: -0.1274

Data Index: 5239698 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1398. Min Act: -0.0950

Data Index: 2496363 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1429. Min Act: -0.0582

Data Index: 2098588 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 1.2254. Min Range: -1.2254

Max Act: 1.1362. Min Act: -0.0990

Data Index: 1064366 (Open Web Text)

Max Activating Token Index: 1023