Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2432 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.4288. Min Act: -0.1700

Data Index: 6560595 (Open Web Text)

Max Activating Token Index: 370

Text #1

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.1645. Min Act: -0.1700

Data Index: 1711305 (Open Web Text)

Max Activating Token Index: 530

Text #2

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.1286. Min Act: -0.1700

Data Index: 7381657 (Open Web Text)

Max Activating Token Index: 415

Text #3

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.1453. Min Act: -0.1700

Data Index: 257546 (Open Web Text)

Max Activating Token Index: 397

Text #4

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0949. Min Act: -0.1700

Data Index: 7075630 (Open Web Text)

Max Activating Token Index: 429

Text #5

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0736. Min Act: -0.1700

Data Index: 8670734 (Open Web Text)

Max Activating Token Index: 522

Text #6

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0723. Min Act: -0.1700

Data Index: 5930610 (Open Web Text)

Max Activating Token Index: 387

Text #7

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0787. Min Act: -0.1700

Data Index: 6906273 (Open Web Text)

Max Activating Token Index: 295

Text #8

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0477. Min Act: -0.1700

Data Index: 3227103 (Open Web Text)

Max Activating Token Index: 895

Text #9

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9994. Min Act: -0.1700

Data Index: 2955048 (Open Web Text)

Max Activating Token Index: 327

Text #10

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0468. Min Act: -0.1700

Data Index: 7654010 (Open Web Text)

Max Activating Token Index: 861

Text #11

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0146. Min Act: -0.1700

Data Index: 2976191 (Open Web Text)

Max Activating Token Index: 104

Text #12

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9884. Min Act: -0.1700

Data Index: 4621681 (Open Web Text)

Max Activating Token Index: 602

Text #13

Max Range: 3.4288. Min Range: -3.4288

Max Act: 3.0118. Min Act: -0.1700

Data Index: 5687652 (Open Web Text)

Max Activating Token Index: 209

Text #14

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9511. Min Act: -0.1700

Data Index: 5044700 (Open Web Text)

Max Activating Token Index: 739

Text #15

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9767. Min Act: -0.1700

Data Index: 5069265 (Open Web Text)

Max Activating Token Index: 475

Text #16

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9847. Min Act: -0.1700

Data Index: 2187796 (Open Web Text)

Max Activating Token Index: 482

Text #17

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9359. Min Act: -0.1700

Data Index: 923366 (Open Web Text)

Max Activating Token Index: 596

Text #18

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9749. Min Act: -0.1700

Data Index: 2395474 (Open Web Text)

Max Activating Token Index: 655

Text #19

Max Range: 3.4288. Min Range: -3.4288

Max Act: 2.9645. Min Act: -0.1700

Data Index: 4734016 (Open Web Text)

Max Activating Token Index: 616