Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 103 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.5004. Min Act: -0.1700

Data Index: 3377960 (Open Web Text)

Max Activating Token Index: 684

Text #1

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3908. Min Act: -0.1700

Data Index: 3794311 (Open Web Text)

Max Activating Token Index: 478

Text #2

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3811. Min Act: -0.1700

Data Index: 6351378 (Open Web Text)

Max Activating Token Index: 883

Text #3

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3981. Min Act: -0.1700

Data Index: 5906849 (Open Web Text)

Max Activating Token Index: 809

Text #4

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3860. Min Act: -0.1700

Data Index: 5906858 (Open Web Text)

Max Activating Token Index: 650

Text #5

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3642. Min Act: -0.1700

Data Index: 4015109 (Open Web Text)

Max Activating Token Index: 872

Text #6

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3612. Min Act: -0.1700

Data Index: 8070615 (Open Web Text)

Max Activating Token Index: 767

Text #7

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3573. Min Act: -0.1700

Data Index: 2134566 (Open Web Text)

Max Activating Token Index: 700

Text #8

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3528. Min Act: -0.1700

Data Index: 2296701 (Open Web Text)

Max Activating Token Index: 599

Text #9

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3452. Min Act: -0.1700

Data Index: 2296690 (Open Web Text)

Max Activating Token Index: 458

Text #10

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3408. Min Act: -0.1700

Data Index: 1597393 (Open Web Text)

Max Activating Token Index: 572

Text #11

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3605. Min Act: -0.1700

Data Index: 6648675 (Open Web Text)

Max Activating Token Index: 450

Text #12

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3491. Min Act: -0.1700

Data Index: 3794327 (Open Web Text)

Max Activating Token Index: 349

Text #13

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3453. Min Act: -0.1700

Data Index: 2134552 (Open Web Text)

Max Activating Token Index: 593

Text #14

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3410. Min Act: -0.1700

Data Index: 2039539 (Open Web Text)

Max Activating Token Index: 688

Text #15

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3341. Min Act: -0.1700

Data Index: 6757290 (Open Web Text)

Max Activating Token Index: 465

Text #16

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3379. Min Act: -0.1700

Data Index: 3099357 (Open Web Text)

Max Activating Token Index: 429

Text #17

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3290. Min Act: -0.1700

Data Index: 5657909 (Open Web Text)

Max Activating Token Index: 1013

Text #18

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3350. Min Act: -0.1700

Data Index: 3099338 (Open Web Text)

Max Activating Token Index: 765

Text #19

Max Range: 3.5004. Min Range: -3.5004

Max Act: 3.3419. Min Act: -0.1700

Data Index: 4015098 (Open Web Text)

Max Activating Token Index: 229