Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 244 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0519. Min Range: -4.0519

Max Act: 4.0519. Min Act: -0.1700

Data Index: 57064 (Open Web Text)

Max Activating Token Index: 583

Text #1

Max Range: 4.0519. Min Range: -4.0519

Max Act: 4.0236. Min Act: -0.1700

Data Index: 244209 (Open Web Text)

Max Activating Token Index: 218

Text #2

Max Range: 4.0519. Min Range: -4.0519

Max Act: 4.0103. Min Act: -0.1700

Data Index: 2962855 (Open Web Text)

Max Activating Token Index: 590

Text #3

Max Range: 4.0519. Min Range: -4.0519

Max Act: 4.0101. Min Act: -0.1700

Data Index: 3027640 (Open Web Text)

Max Activating Token Index: 336

Text #4

Max Range: 4.0519. Min Range: -4.0519

Max Act: 4.0213. Min Act: -0.1700

Data Index: 478594 (Open Web Text)

Max Activating Token Index: 381

Text #5

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9732. Min Act: -0.1700

Data Index: 926551 (Open Web Text)

Max Activating Token Index: 475

Text #6

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9693. Min Act: -0.1700

Data Index: 5338908 (Open Web Text)

Max Activating Token Index: 518

Text #7

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9268. Min Act: -0.1700

Data Index: 318128 (Open Web Text)

Max Activating Token Index: 170

Text #8

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9573. Min Act: -0.1700

Data Index: 4436914 (Open Web Text)

Max Activating Token Index: 215

Text #9

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9499. Min Act: -0.1700

Data Index: 8309098 (Open Web Text)

Max Activating Token Index: 504

Text #10

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9421. Min Act: -0.1700

Data Index: 4250045 (Open Web Text)

Max Activating Token Index: 862

Text #11

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9338. Min Act: -0.1700

Data Index: 3820269 (Open Web Text)

Max Activating Token Index: 1020

Text #12

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9445. Min Act: -0.1700

Data Index: 3563919 (Open Web Text)

Max Activating Token Index: 44

Text #13

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9323. Min Act: -0.1700

Data Index: 7608748 (Open Web Text)

Max Activating Token Index: 677

Text #14

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9444. Min Act: -0.1700

Data Index: 5954597 (Open Web Text)

Max Activating Token Index: 672

Text #15

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9373. Min Act: -0.1700

Data Index: 4929323 (Open Web Text)

Max Activating Token Index: 447

Text #16

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9536. Min Act: -0.1700

Data Index: 1879263 (Open Web Text)

Max Activating Token Index: 161

Text #17

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9388. Min Act: -0.1700

Data Index: 4703676 (Open Web Text)

Max Activating Token Index: 706

Text #18

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9469. Min Act: -0.1700

Data Index: 2645499 (Open Web Text)

Max Activating Token Index: 299

Text #19

Max Range: 4.0519. Min Range: -4.0519

Max Act: 3.9261. Min Act: -0.1700

Data Index: 2460331 (Open Web Text)

Max Activating Token Index: 538