Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1031 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2328. Min Range: -4.2328

Max Act: 4.2328. Min Act: -0.1700

Data Index: 1048347 (Open Web Text)

Max Activating Token Index: 771

Text #1

Max Range: 4.2328. Min Range: -4.2328

Max Act: 4.1754. Min Act: -0.1700

Data Index: 6897840 (Open Web Text)

Max Activating Token Index: 783

Text #2

Max Range: 4.2328. Min Range: -4.2328

Max Act: 4.1391. Min Act: -0.1700

Data Index: 6342308 (Open Web Text)

Max Activating Token Index: 846

Text #3

Max Range: 4.2328. Min Range: -4.2328

Max Act: 4.0726. Min Act: -0.1700

Data Index: 5331407 (Open Web Text)

Max Activating Token Index: 361

Text #4

Max Range: 4.2328. Min Range: -4.2328

Max Act: 4.0191. Min Act: -0.1700

Data Index: 5692899 (Open Web Text)

Max Activating Token Index: 48

Text #5

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9904. Min Act: -0.1700

Data Index: 6667221 (Open Web Text)

Max Activating Token Index: 542

Text #6

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9899. Min Act: -0.1700

Data Index: 70378 (Open Web Text)

Max Activating Token Index: 599

Text #7

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9831. Min Act: -0.1700

Data Index: 4981160 (Open Web Text)

Max Activating Token Index: 239

Text #8

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9524. Min Act: -0.1700

Data Index: 4581406 (Open Web Text)

Max Activating Token Index: 629

Text #9

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9627. Min Act: -0.1700

Data Index: 7759469 (Open Web Text)

Max Activating Token Index: 124

Text #10

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9486. Min Act: -0.1700

Data Index: 3116389 (Open Web Text)

Max Activating Token Index: 933

Text #11

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9626. Min Act: -0.1700

Data Index: 5131070 (Open Web Text)

Max Activating Token Index: 826

Text #12

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9234. Min Act: -0.1700

Data Index: 2603200 (Open Web Text)

Max Activating Token Index: 462

Text #13

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9242. Min Act: -0.1700

Data Index: 3919467 (Open Web Text)

Max Activating Token Index: 614

Text #14

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9340. Min Act: -0.1700

Data Index: 6017811 (Open Web Text)

Max Activating Token Index: 986

Text #15

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9339. Min Act: -0.1700

Data Index: 1522259 (Open Web Text)

Max Activating Token Index: 112

Text #16

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.8985. Min Act: -0.1700

Data Index: 2536058 (Open Web Text)

Max Activating Token Index: 1014

Text #17

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9414. Min Act: -0.1700

Data Index: 5588422 (Open Web Text)

Max Activating Token Index: 955

Text #18

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9302. Min Act: -0.1700

Data Index: 1305666 (Open Web Text)

Max Activating Token Index: 972

Text #19

Max Range: 4.2328. Min Range: -4.2328

Max Act: 3.9317. Min Act: -0.1700

Data Index: 4868744 (Open Web Text)

Max Activating Token Index: 32