Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 646 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.3257. Min Act: -0.1700

Data Index: 6907106 (Open Web Text)

Max Activating Token Index: 522

Text #1

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.3156. Min Act: -0.1700

Data Index: 1552344 (Open Web Text)

Max Activating Token Index: 567

Text #2

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2966. Min Act: -0.1700

Data Index: 673999 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2882. Min Act: -0.1700

Data Index: 5730022 (Open Web Text)

Max Activating Token Index: 417

Text #4

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2853. Min Act: -0.1700

Data Index: 5274947 (Open Web Text)

Max Activating Token Index: 7

Text #5

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2947. Min Act: -0.1700

Data Index: 6955755 (Open Web Text)

Max Activating Token Index: 714

Text #6

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2736. Min Act: -0.1700

Data Index: 6612508 (Open Web Text)

Max Activating Token Index: 402

Text #7

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2721. Min Act: -0.1700

Data Index: 5813290 (Open Web Text)

Max Activating Token Index: 157

Text #8

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2465. Min Act: -0.1700

Data Index: 4416336 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2503. Min Act: -0.1700

Data Index: 7707213 (Open Web Text)

Max Activating Token Index: 15

Text #10

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2558. Min Act: -0.1700

Data Index: 6675827 (Open Web Text)

Max Activating Token Index: 6

Text #11

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2508. Min Act: -0.1700

Data Index: 7349856 (Open Web Text)

Max Activating Token Index: 30

Text #12

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2607. Min Act: -0.1700

Data Index: 2199598 (Open Web Text)

Max Activating Token Index: 306

Text #13

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2530. Min Act: -0.1700

Data Index: 6898988 (Open Web Text)

Max Activating Token Index: 25

Text #14

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2511. Min Act: -0.1700

Data Index: 3235822 (Open Web Text)

Max Activating Token Index: 7

Text #15

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2528. Min Act: -0.1700

Data Index: 3184706 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2613. Min Act: -0.1700

Data Index: 2898711 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2534. Min Act: -0.1700

Data Index: 454383 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2311. Min Act: -0.1700

Data Index: 422944 (Open Web Text)

Max Activating Token Index: 12

Text #19

Max Range: 2.3257. Min Range: -2.3257

Max Act: 2.2377. Min Act: -0.1700

Data Index: 5283907 (Open Web Text)

Max Activating Token Index: 5