Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2888 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.8477. Min Act: -0.1700

Data Index: 5677237 (Open Web Text)

Max Activating Token Index: 137

Text #1

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.7648. Min Act: -0.1700

Data Index: 167147 (Open Web Text)

Max Activating Token Index: 425

Text #2

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.7460. Min Act: -0.1700

Data Index: 5578791 (Open Web Text)

Max Activating Token Index: 386

Text #3

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6665. Min Act: -0.1700

Data Index: 8596216 (Open Web Text)

Max Activating Token Index: 637

Text #4

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6091. Min Act: -0.1700

Data Index: 4981324 (Open Web Text)

Max Activating Token Index: 630

Text #5

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6547. Min Act: -0.1700

Data Index: 8153337 (Open Web Text)

Max Activating Token Index: 405

Text #6

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6513. Min Act: -0.1700

Data Index: 6297349 (Open Web Text)

Max Activating Token Index: 613

Text #7

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6410. Min Act: -0.1700

Data Index: 2900269 (Open Web Text)

Max Activating Token Index: 443

Text #8

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6300. Min Act: -0.1700

Data Index: 1452604 (Open Web Text)

Max Activating Token Index: 143

Text #9

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5836. Min Act: -0.1700

Data Index: 7553509 (Open Web Text)

Max Activating Token Index: 381

Text #10

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.6027. Min Act: -0.1700

Data Index: 8778547 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.4938. Min Act: -0.1700

Data Index: 3337591 (Open Web Text)

Max Activating Token Index: 353

Text #12

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5239. Min Act: -0.1700

Data Index: 2397691 (Open Web Text)

Max Activating Token Index: 303

Text #13

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5740. Min Act: -0.1700

Data Index: 5406560 (Open Web Text)

Max Activating Token Index: 603

Text #14

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5596. Min Act: -0.1700

Data Index: 90473 (Open Web Text)

Max Activating Token Index: 96

Text #15

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5519. Min Act: -0.1700

Data Index: 4206056 (Open Web Text)

Max Activating Token Index: 990

Text #16

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5276. Min Act: -0.1700

Data Index: 6747099 (Open Web Text)

Max Activating Token Index: 483

Text #17

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5312. Min Act: -0.1700

Data Index: 8041745 (Open Web Text)

Max Activating Token Index: 64

Text #18

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.5474. Min Act: -0.1700

Data Index: 3213157 (Open Web Text)

Max Activating Token Index: 752

Text #19

Max Range: 3.8477. Min Range: -3.8477

Max Act: 3.4840. Min Act: -0.1700

Data Index: 8239800 (Open Web Text)

Max Activating Token Index: 236