Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 431 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.3427. Min Act: -0.1700

Data Index: 7228770 (Open Web Text)

Max Activating Token Index: 495

Text #1

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.2416. Min Act: -0.1700

Data Index: 366683 (Open Web Text)

Max Activating Token Index: 462

Text #2

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.2087. Min Act: -0.1700

Data Index: 4771376 (Open Web Text)

Max Activating Token Index: 446

Text #3

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.2137. Min Act: -0.1700

Data Index: 3891922 (Open Web Text)

Max Activating Token Index: 475

Text #4

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.2187. Min Act: -0.1700

Data Index: 5233537 (Open Web Text)

Max Activating Token Index: 732

Text #5

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.2228. Min Act: -0.1700

Data Index: 8224482 (Open Web Text)

Max Activating Token Index: 269

Text #6

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1802. Min Act: -0.1700

Data Index: 3928439 (Open Web Text)

Max Activating Token Index: 575

Text #7

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1691. Min Act: -0.1700

Data Index: 6505655 (Open Web Text)

Max Activating Token Index: 984

Text #8

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1885. Min Act: -0.1700

Data Index: 343719 (Open Web Text)

Max Activating Token Index: 527

Text #9

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1921. Min Act: -0.1700

Data Index: 6690162 (Open Web Text)

Max Activating Token Index: 452

Text #10

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1781. Min Act: -0.1700

Data Index: 4878610 (Open Web Text)

Max Activating Token Index: 627

Text #11

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1676. Min Act: -0.1700

Data Index: 2301018 (Open Web Text)

Max Activating Token Index: 488

Text #12

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1500. Min Act: -0.1700

Data Index: 3250576 (Open Web Text)

Max Activating Token Index: 535

Text #13

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1554. Min Act: -0.1700

Data Index: 3770900 (Open Web Text)

Max Activating Token Index: 738

Text #14

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1471. Min Act: -0.1700

Data Index: 5762363 (Open Web Text)

Max Activating Token Index: 869

Text #15

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1416. Min Act: -0.1700

Data Index: 1426043 (Open Web Text)

Max Activating Token Index: 23

Text #16

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1523. Min Act: -0.1700

Data Index: 6871553 (Open Web Text)

Max Activating Token Index: 740

Text #17

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1438. Min Act: -0.1700

Data Index: 2435720 (Open Web Text)

Max Activating Token Index: 508

Text #18

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1585. Min Act: -0.1700

Data Index: 7680466 (Open Web Text)

Max Activating Token Index: 877

Text #19

Max Range: 4.3427. Min Range: -4.3427

Max Act: 4.1358. Min Act: -0.1700

Data Index: 4210484 (Open Web Text)

Max Activating Token Index: 669