Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1585 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.5829. Min Act: -0.1700

Data Index: 633614 (Open Web Text)

Max Activating Token Index: 575

Text #1

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.5956. Min Act: -0.1700

Data Index: 4663508 (Open Web Text)

Max Activating Token Index: 316

Text #2

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.5445. Min Act: -0.1700

Data Index: 5710553 (Open Web Text)

Max Activating Token Index: 625

Text #3

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4846. Min Act: -0.1700

Data Index: 7211839 (Open Web Text)

Max Activating Token Index: 509

Text #4

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4845. Min Act: -0.1700

Data Index: 2540739 (Open Web Text)

Max Activating Token Index: 773

Text #5

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4522. Min Act: -0.1700

Data Index: 4161695 (Open Web Text)

Max Activating Token Index: 561

Text #6

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4420. Min Act: -0.1700

Data Index: 1486109 (Open Web Text)

Max Activating Token Index: 309

Text #7

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4251. Min Act: -0.1700

Data Index: 2212902 (Open Web Text)

Max Activating Token Index: 622

Text #8

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4223. Min Act: -0.1700

Data Index: 6438197 (Open Web Text)

Max Activating Token Index: 303

Text #9

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.4115. Min Act: -0.1700

Data Index: 1430506 (Open Web Text)

Max Activating Token Index: 817

Text #10

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3962. Min Act: -0.1700

Data Index: 135931 (Open Web Text)

Max Activating Token Index: 506

Text #11

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3833. Min Act: -0.1700

Data Index: 6360280 (Open Web Text)

Max Activating Token Index: 287

Text #12

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3611. Min Act: -0.1700

Data Index: 1461300 (Open Web Text)

Max Activating Token Index: 674

Text #13

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3560. Min Act: -0.1700

Data Index: 6209077 (Open Web Text)

Max Activating Token Index: 185

Text #14

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3570. Min Act: -0.1700

Data Index: 5449711 (Open Web Text)

Max Activating Token Index: 860

Text #15

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3389. Min Act: -0.1700

Data Index: 1329214 (Open Web Text)

Max Activating Token Index: 580

Text #16

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3604. Min Act: -0.1700

Data Index: 78439 (Open Web Text)

Max Activating Token Index: 526

Text #17

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3404. Min Act: -0.1700

Data Index: 2618994 (Open Web Text)

Max Activating Token Index: 571

Text #18

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3398. Min Act: -0.1700

Data Index: 6639857 (Open Web Text)

Max Activating Token Index: 421

Text #19

Max Range: 3.5956. Min Range: -3.5956

Max Act: 3.3531. Min Act: -0.1700

Data Index: 1465960 (Open Web Text)

Max Activating Token Index: 505