Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1833 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.9953. Min Act: -0.1700

Data Index: 1562241 (Open Web Text)

Max Activating Token Index: 887

Text #1

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.9246. Min Act: -0.1700

Data Index: 4117158 (Open Web Text)

Max Activating Token Index: 444

Text #2

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8687. Min Act: -0.1700

Data Index: 7954751 (Open Web Text)

Max Activating Token Index: 618

Text #3

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8199. Min Act: -0.1700

Data Index: 561325 (Open Web Text)

Max Activating Token Index: 34

Text #4

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8090. Min Act: -0.1700

Data Index: 754775 (Open Web Text)

Max Activating Token Index: 939

Text #5

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8093. Min Act: -0.1700

Data Index: 3068718 (Open Web Text)

Max Activating Token Index: 786

Text #6

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8227. Min Act: -0.1700

Data Index: 4144958 (Open Web Text)

Max Activating Token Index: 528

Text #7

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8228. Min Act: -0.1700

Data Index: 4300498 (Open Web Text)

Max Activating Token Index: 894

Text #8

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7580. Min Act: -0.1700

Data Index: 3050081 (Open Web Text)

Max Activating Token Index: 578

Text #9

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.8291. Min Act: -0.1700

Data Index: 3297287 (Open Web Text)

Max Activating Token Index: 513

Text #10

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7846. Min Act: -0.1700

Data Index: 3391588 (Open Web Text)

Max Activating Token Index: 367

Text #11

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7462. Min Act: -0.1700

Data Index: 3517191 (Open Web Text)

Max Activating Token Index: 449

Text #12

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7160. Min Act: -0.1700

Data Index: 5202289 (Open Web Text)

Max Activating Token Index: 813

Text #13

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7068. Min Act: -0.1700

Data Index: 3806470 (Open Web Text)

Max Activating Token Index: 325

Text #14

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7027. Min Act: -0.1700

Data Index: 3923781 (Open Web Text)

Max Activating Token Index: 317

Text #15

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.6821. Min Act: -0.1700

Data Index: 6424687 (Open Web Text)

Max Activating Token Index: 181

Text #16

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.6557. Min Act: -0.1700

Data Index: 5232830 (Open Web Text)

Max Activating Token Index: 173

Text #17

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7303. Min Act: -0.1700

Data Index: 3574911 (Open Web Text)

Max Activating Token Index: 756

Text #18

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.6838. Min Act: -0.1700

Data Index: 209330 (Open Web Text)

Max Activating Token Index: 234

Text #19

Max Range: 4.9953. Min Range: -4.9953

Max Act: 4.7325. Min Act: -0.1700

Data Index: 1373339 (Open Web Text)

Max Activating Token Index: 571