Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 86 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.1158. Min Act: -0.1700

Data Index: 1878753 (Open Web Text)

Max Activating Token Index: 521

Text #1

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.1192. Min Act: -0.1700

Data Index: 5789154 (Open Web Text)

Max Activating Token Index: 928

Text #2

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0696. Min Act: -0.1700

Data Index: 2497227 (Open Web Text)

Max Activating Token Index: 276

Text #3

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0793. Min Act: -0.1700

Data Index: 1440557 (Open Web Text)

Max Activating Token Index: 787

Text #4

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0703. Min Act: -0.1700

Data Index: 758253 (Open Web Text)

Max Activating Token Index: 424

Text #5

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0653. Min Act: -0.1700

Data Index: 1429372 (Open Web Text)

Max Activating Token Index: 27

Text #6

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0444. Min Act: -0.1700

Data Index: 2581590 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0472. Min Act: -0.1700

Data Index: 6524216 (Open Web Text)

Max Activating Token Index: 269

Text #8

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0524. Min Act: -0.1700

Data Index: 7600110 (Open Web Text)

Max Activating Token Index: 64

Text #9

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0345. Min Act: -0.1700

Data Index: 5352188 (Open Web Text)

Max Activating Token Index: 948

Text #10

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0393. Min Act: -0.1700

Data Index: 1685335 (Open Web Text)

Max Activating Token Index: 20

Text #11

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0263. Min Act: -0.1700

Data Index: 3069611 (Open Web Text)

Max Activating Token Index: 791

Text #12

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0352. Min Act: -0.1700

Data Index: 2245323 (Open Web Text)

Max Activating Token Index: 989

Text #13

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0433. Min Act: -0.1700

Data Index: 8826986 (Open Web Text)

Max Activating Token Index: 918

Text #14

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0335. Min Act: -0.1700

Data Index: 3893092 (Open Web Text)

Max Activating Token Index: 122

Text #15

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0330. Min Act: -0.1700

Data Index: 2778533 (Open Web Text)

Max Activating Token Index: 370

Text #16

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0280. Min Act: -0.1700

Data Index: 2186756 (Open Web Text)

Max Activating Token Index: 70

Text #17

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0228. Min Act: -0.1700

Data Index: 4575300 (Open Web Text)

Max Activating Token Index: 202

Text #18

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0242. Min Act: -0.1700

Data Index: 3441385 (Open Web Text)

Max Activating Token Index: 120

Text #19

Max Range: 3.1192. Min Range: -3.1192

Max Act: 3.0178. Min Act: -0.1700

Data Index: 359873 (Open Web Text)

Max Activating Token Index: 270