Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2060 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.1351. Min Act: -0.1700

Data Index: 5087243 (Open Web Text)

Max Activating Token Index: 952

Text #1

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.1517. Min Act: -0.1700

Data Index: 2874475 (Open Web Text)

Max Activating Token Index: 878

Text #2

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.1091. Min Act: -0.1700

Data Index: 3009272 (Open Web Text)

Max Activating Token Index: 974

Text #3

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0917. Min Act: -0.1700

Data Index: 7580702 (Open Web Text)

Max Activating Token Index: 637

Text #4

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0826. Min Act: -0.1700

Data Index: 6441474 (Open Web Text)

Max Activating Token Index: 678

Text #5

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0705. Min Act: -0.1700

Data Index: 2577794 (Open Web Text)

Max Activating Token Index: 260

Text #6

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0267. Min Act: -0.1700

Data Index: 2071673 (Open Web Text)

Max Activating Token Index: 237

Text #7

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0378. Min Act: -0.1700

Data Index: 3464348 (Open Web Text)

Max Activating Token Index: 899

Text #8

Max Range: 3.1517. Min Range: -3.1517

Max Act: 3.0121. Min Act: -0.1700

Data Index: 6837337 (Open Web Text)

Max Activating Token Index: 834

Text #9

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9993. Min Act: -0.1700

Data Index: 8001292 (Open Web Text)

Max Activating Token Index: 654

Text #10

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9906. Min Act: -0.1700

Data Index: 7160341 (Open Web Text)

Max Activating Token Index: 690

Text #11

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9513. Min Act: -0.1700

Data Index: 8797383 (Open Web Text)

Max Activating Token Index: 279

Text #12

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9404. Min Act: -0.1700

Data Index: 4321207 (Open Web Text)

Max Activating Token Index: 634

Text #13

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9444. Min Act: -0.1700

Data Index: 7800450 (Open Web Text)

Max Activating Token Index: 507

Text #14

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9182. Min Act: -0.1700

Data Index: 5255780 (Open Web Text)

Max Activating Token Index: 805

Text #15

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9316. Min Act: -0.1700

Data Index: 7165654 (Open Web Text)

Max Activating Token Index: 899

Text #16

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9355. Min Act: -0.1700

Data Index: 3731304 (Open Web Text)

Max Activating Token Index: 749

Text #17

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9197. Min Act: -0.1700

Data Index: 6534460 (Open Web Text)

Max Activating Token Index: 172

Text #18

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9061. Min Act: -0.1700

Data Index: 6501965 (Open Web Text)

Max Activating Token Index: 973

Text #19

Max Range: 3.1517. Min Range: -3.1517

Max Act: 2.9212. Min Act: -0.1700

Data Index: 7262020 (Open Web Text)

Max Activating Token Index: 468