Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1889 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.8196. Min Act: -0.1700

Data Index: 71350 (Open Web Text)

Max Activating Token Index: 70

Text #1

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.6519. Min Act: -0.1700

Data Index: 2496349 (Open Web Text)

Max Activating Token Index: 449

Text #2

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.6298. Min Act: -0.1700

Data Index: 2389891 (Open Web Text)

Max Activating Token Index: 1020

Text #3

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.5806. Min Act: -0.1700

Data Index: 2778278 (Open Web Text)

Max Activating Token Index: 639

Text #4

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.5938. Min Act: -0.1700

Data Index: 1233198 (Open Web Text)

Max Activating Token Index: 844

Text #5

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.5531. Min Act: -0.1700

Data Index: 6534028 (Open Web Text)

Max Activating Token Index: 214

Text #6

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.5064. Min Act: -0.1700

Data Index: 8321160 (Open Web Text)

Max Activating Token Index: 444

Text #7

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4955. Min Act: -0.1700

Data Index: 3337913 (Open Web Text)

Max Activating Token Index: 731

Text #8

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4229. Min Act: -0.1700

Data Index: 3917974 (Open Web Text)

Max Activating Token Index: 906

Text #9

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4474. Min Act: -0.1700

Data Index: 1032092 (Open Web Text)

Max Activating Token Index: 78

Text #10

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4618. Min Act: -0.1700

Data Index: 7191283 (Open Web Text)

Max Activating Token Index: 104

Text #11

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4499. Min Act: -0.1700

Data Index: 7748681 (Open Web Text)

Max Activating Token Index: 290

Text #12

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4452. Min Act: -0.1700

Data Index: 7237922 (Open Web Text)

Max Activating Token Index: 26

Text #13

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4199. Min Act: -0.1700

Data Index: 5491872 (Open Web Text)

Max Activating Token Index: 962

Text #14

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4508. Min Act: -0.1700

Data Index: 3337915 (Open Web Text)

Max Activating Token Index: 996

Text #15

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4271. Min Act: -0.1700

Data Index: 7110161 (Open Web Text)

Max Activating Token Index: 96

Text #16

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.4094. Min Act: -0.1700

Data Index: 1298226 (Open Web Text)

Max Activating Token Index: 952

Text #17

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.3868. Min Act: -0.1700

Data Index: 239531 (Open Web Text)

Max Activating Token Index: 187

Text #18

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.3970. Min Act: -0.1700

Data Index: 1572163 (Open Web Text)

Max Activating Token Index: 873

Text #19

Max Range: 3.8196. Min Range: -3.8196

Max Act: 3.3649. Min Act: -0.1700

Data Index: 2260690 (Open Web Text)

Max Activating Token Index: 193