Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1916 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0399. Min Range: -4.0399

Max Act: 4.0399. Min Act: -0.1700

Data Index: 6578361 (Open Web Text)

Max Activating Token Index: 126

Text #1

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9414. Min Act: -0.1700

Data Index: 5112274 (Open Web Text)

Max Activating Token Index: 40

Text #2

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9216. Min Act: -0.1700

Data Index: 2907017 (Open Web Text)

Max Activating Token Index: 743

Text #3

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.9349. Min Act: -0.1700

Data Index: 6571153 (Open Web Text)

Max Activating Token Index: 670

Text #4

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8607. Min Act: -0.1700

Data Index: 1841280 (Open Web Text)

Max Activating Token Index: 307

Text #5

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8518. Min Act: -0.1700

Data Index: 4575000 (Open Web Text)

Max Activating Token Index: 620

Text #6

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7887. Min Act: -0.1700

Data Index: 4125838 (Open Web Text)

Max Activating Token Index: 367

Text #7

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8231. Min Act: -0.1700

Data Index: 8096892 (Open Web Text)

Max Activating Token Index: 45

Text #8

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7863. Min Act: -0.1700

Data Index: 4864115 (Open Web Text)

Max Activating Token Index: 955

Text #9

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.8089. Min Act: -0.1700

Data Index: 1976325 (Open Web Text)

Max Activating Token Index: 644

Text #10

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7690. Min Act: -0.1700

Data Index: 8524927 (Open Web Text)

Max Activating Token Index: 25

Text #11

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7748. Min Act: -0.1700

Data Index: 4836049 (Open Web Text)

Max Activating Token Index: 140

Text #12

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7603. Min Act: -0.1700

Data Index: 5062611 (Open Web Text)

Max Activating Token Index: 533

Text #13

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7350. Min Act: -0.1700

Data Index: 511844 (Open Web Text)

Max Activating Token Index: 157

Text #14

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7251. Min Act: -0.1700

Data Index: 6772844 (Open Web Text)

Max Activating Token Index: 828

Text #15

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7175. Min Act: -0.1700

Data Index: 2626411 (Open Web Text)

Max Activating Token Index: 991

Text #16

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7206. Min Act: -0.1700

Data Index: 1208343 (Open Web Text)

Max Activating Token Index: 956

Text #17

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.7172. Min Act: -0.1700

Data Index: 1537794 (Open Web Text)

Max Activating Token Index: 869

Text #18

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.6887. Min Act: -0.1700

Data Index: 5747109 (Open Web Text)

Max Activating Token Index: 572

Text #19

Max Range: 4.0399. Min Range: -4.0399

Max Act: 3.6924. Min Act: -0.1700

Data Index: 481619 (Open Web Text)

Max Activating Token Index: 785