Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1038 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.1398. Min Act: -0.1700

Data Index: 7307732 (Open Web Text)

Max Activating Token Index: 370

Text #1

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.1126. Min Act: -0.1700

Data Index: 534808 (Open Web Text)

Max Activating Token Index: 917

Text #2

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0595. Min Act: -0.1700

Data Index: 3365920 (Open Web Text)

Max Activating Token Index: 512

Text #3

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0842. Min Act: -0.1700

Data Index: 755762 (Open Web Text)

Max Activating Token Index: 791

Text #4

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0469. Min Act: -0.1700

Data Index: 6966335 (Open Web Text)

Max Activating Token Index: 659

Text #5

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0355. Min Act: -0.1700

Data Index: 6242459 (Open Web Text)

Max Activating Token Index: 823

Text #6

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0021. Min Act: -0.1700

Data Index: 418560 (Open Web Text)

Max Activating Token Index: 346

Text #7

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0315. Min Act: -0.1700

Data Index: 2625568 (Open Web Text)

Max Activating Token Index: 453

Text #8

Max Range: 5.1398. Min Range: -5.1398

Max Act: 5.0315. Min Act: -0.1700

Data Index: 3141493 (Open Web Text)

Max Activating Token Index: 229

Text #9

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9812. Min Act: -0.1700

Data Index: 7373732 (Open Web Text)

Max Activating Token Index: 217

Text #10

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9949. Min Act: -0.1700

Data Index: 5958822 (Open Web Text)

Max Activating Token Index: 869

Text #11

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9900. Min Act: -0.1700

Data Index: 3316484 (Open Web Text)

Max Activating Token Index: 969

Text #12

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9736. Min Act: -0.1700

Data Index: 7586251 (Open Web Text)

Max Activating Token Index: 588

Text #13

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9809. Min Act: -0.1700

Data Index: 4779478 (Open Web Text)

Max Activating Token Index: 279

Text #14

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9505. Min Act: -0.1700

Data Index: 375715 (Open Web Text)

Max Activating Token Index: 234

Text #15

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9492. Min Act: -0.1700

Data Index: 6908505 (Open Web Text)

Max Activating Token Index: 866

Text #16

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9464. Min Act: -0.1700

Data Index: 5448189 (Open Web Text)

Max Activating Token Index: 646

Text #17

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9670. Min Act: -0.1700

Data Index: 305039 (Open Web Text)

Max Activating Token Index: 337

Text #18

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9530. Min Act: -0.1700

Data Index: 6598411 (Open Web Text)

Max Activating Token Index: 833

Text #19

Max Range: 5.1398. Min Range: -5.1398

Max Act: 4.9866. Min Act: -0.1699

Data Index: 5535355 (Open Web Text)

Max Activating Token Index: 979