Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2366 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.2317. Min Act: -0.1700

Data Index: 2069242 (Open Web Text)

Max Activating Token Index: 2

Text #1

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.2332. Min Act: -0.1700

Data Index: 1944479 (Open Web Text)

Max Activating Token Index: 130

Text #2

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1936. Min Act: -0.1700

Data Index: 3288633 (Open Web Text)

Max Activating Token Index: 17

Text #3

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1998. Min Act: -0.1700

Data Index: 2852462 (Open Web Text)

Max Activating Token Index: 588

Text #4

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1970. Min Act: -0.1700

Data Index: 7713683 (Open Web Text)

Max Activating Token Index: 46

Text #5

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1961. Min Act: -0.1700

Data Index: 3355617 (Open Web Text)

Max Activating Token Index: 258

Text #6

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1869. Min Act: -0.1700

Data Index: 1875208 (Open Web Text)

Max Activating Token Index: 474

Text #7

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1878. Min Act: -0.1700

Data Index: 170669 (Open Web Text)

Max Activating Token Index: 505

Text #8

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1862. Min Act: -0.1700

Data Index: 4349587 (Open Web Text)

Max Activating Token Index: 16

Text #9

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1657. Min Act: -0.1700

Data Index: 6269326 (Open Web Text)

Max Activating Token Index: 15

Text #10

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1654. Min Act: -0.1700

Data Index: 5185131 (Open Web Text)

Max Activating Token Index: 583

Text #11

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1768. Min Act: -0.1700

Data Index: 3850248 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1889. Min Act: -0.1700

Data Index: 5213653 (Open Web Text)

Max Activating Token Index: 320

Text #13

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1653. Min Act: -0.1700

Data Index: 3123418 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1880. Min Act: -0.1700

Data Index: 3797064 (Open Web Text)

Max Activating Token Index: 683

Text #15

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1771. Min Act: -0.1700

Data Index: 3395588 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1767. Min Act: -0.1700

Data Index: 1013175 (Open Web Text)

Max Activating Token Index: 635

Text #17

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1806. Min Act: -0.1700

Data Index: 5611619 (Open Web Text)

Max Activating Token Index: 1022

Text #18

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1442. Min Act: -0.1700

Data Index: 4242856 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 2.2332. Min Range: -2.2332

Max Act: 2.1433. Min Act: -0.1700

Data Index: 2123902 (Open Web Text)

Max Activating Token Index: 4