Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 180 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.4716. Min Act: -0.1700

Data Index: 2355496 (Open Web Text)

Max Activating Token Index: 308

Text #1

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3799. Min Act: -0.1700

Data Index: 6199475 (Open Web Text)

Max Activating Token Index: 441

Text #2

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.4110. Min Act: -0.1700

Data Index: 2335689 (Open Web Text)

Max Activating Token Index: 401

Text #3

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3590. Min Act: -0.1700

Data Index: 2474029 (Open Web Text)

Max Activating Token Index: 796

Text #4

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3132. Min Act: -0.1700

Data Index: 6500407 (Open Web Text)

Max Activating Token Index: 598

Text #5

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3274. Min Act: -0.1700

Data Index: 1009674 (Open Web Text)

Max Activating Token Index: 262

Text #6

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3008. Min Act: -0.1700

Data Index: 8256344 (Open Web Text)

Max Activating Token Index: 536

Text #7

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2949. Min Act: -0.1700

Data Index: 7902789 (Open Web Text)

Max Activating Token Index: 541

Text #8

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.3113. Min Act: -0.1700

Data Index: 2139816 (Open Web Text)

Max Activating Token Index: 473

Text #9

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2854. Min Act: -0.1700

Data Index: 8154650 (Open Web Text)

Max Activating Token Index: 342

Text #10

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2897. Min Act: -0.1700

Data Index: 6106454 (Open Web Text)

Max Activating Token Index: 742

Text #11

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2855. Min Act: -0.1700

Data Index: 6873643 (Open Web Text)

Max Activating Token Index: 265

Text #12

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2671. Min Act: -0.1700

Data Index: 2568754 (Open Web Text)

Max Activating Token Index: 928

Text #13

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2743. Min Act: -0.1700

Data Index: 7998896 (Open Web Text)

Max Activating Token Index: 809

Text #14

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2484. Min Act: -0.1700

Data Index: 8521347 (Open Web Text)

Max Activating Token Index: 866

Text #15

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2541. Min Act: -0.1700

Data Index: 507565 (Open Web Text)

Max Activating Token Index: 480

Text #16

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2622. Min Act: -0.1700

Data Index: 1912215 (Open Web Text)

Max Activating Token Index: 446

Text #17

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2654. Min Act: -0.1700

Data Index: 1845806 (Open Web Text)

Max Activating Token Index: 462

Text #18

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2546. Min Act: -0.1700

Data Index: 4327395 (Open Web Text)

Max Activating Token Index: 363

Text #19

Max Range: 3.4716. Min Range: -3.4716

Max Act: 3.2588. Min Act: -0.1700

Data Index: 6905340 (Open Web Text)

Max Activating Token Index: 958