Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3017 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.5554. Min Act: -0.1700

Data Index: 6214505 (Open Web Text)

Max Activating Token Index: 849

Text #1

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.5273. Min Act: -0.1700

Data Index: 6214490 (Open Web Text)

Max Activating Token Index: 559

Text #2

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.4589. Min Act: -0.1700

Data Index: 6214487 (Open Web Text)

Max Activating Token Index: 213

Text #3

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3858. Min Act: -0.1700

Data Index: 6214488 (Open Web Text)

Max Activating Token Index: 307

Text #4

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3553. Min Act: -0.1700

Data Index: 6214509 (Open Web Text)

Max Activating Token Index: 148

Text #5

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3367. Min Act: -0.1700

Data Index: 2218254 (Open Web Text)

Max Activating Token Index: 141

Text #6

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3559. Min Act: -0.1700

Data Index: 5876405 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3361. Min Act: -0.1700

Data Index: 8289983 (Open Web Text)

Max Activating Token Index: 637

Text #8

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3283. Min Act: -0.1700

Data Index: 6326991 (Open Web Text)

Max Activating Token Index: 93

Text #9

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.3000. Min Act: -0.1700

Data Index: 6214508 (Open Web Text)

Max Activating Token Index: 934

Text #10

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2928. Min Act: -0.1700

Data Index: 4676712 (Open Web Text)

Max Activating Token Index: 548

Text #11

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2576. Min Act: -0.1700

Data Index: 3414546 (Open Web Text)

Max Activating Token Index: 904

Text #12

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2670. Min Act: -0.1700

Data Index: 6129836 (Open Web Text)

Max Activating Token Index: 659

Text #13

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2137. Min Act: -0.1700

Data Index: 3034260 (Open Web Text)

Max Activating Token Index: 1010

Text #14

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2141. Min Act: -0.1700

Data Index: 171442 (Open Web Text)

Max Activating Token Index: 273

Text #15

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.1910. Min Act: -0.1700

Data Index: 8048216 (Open Web Text)

Max Activating Token Index: 281

Text #16

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.1630. Min Act: -0.1700

Data Index: 6772588 (Open Web Text)

Max Activating Token Index: 755

Text #17

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.2148. Min Act: -0.1700

Data Index: 8517236 (Open Web Text)

Max Activating Token Index: 63

Text #18

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.1762. Min Act: -0.1700

Data Index: 7973746 (Open Web Text)

Max Activating Token Index: 986

Text #19

Max Range: 3.5554. Min Range: -3.5554

Max Act: 3.1887. Min Act: -0.1700

Data Index: 4403624 (Open Web Text)

Max Activating Token Index: 39