Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1145 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.3991. Min Act: -0.1700

Data Index: 1735836 (Open Web Text)

Max Activating Token Index: 289

Text #1

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.3423. Min Act: -0.1700

Data Index: 1735835 (Open Web Text)

Max Activating Token Index: 697

Text #2

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.3152. Min Act: -0.1700

Data Index: 7494480 (Open Web Text)

Max Activating Token Index: 44

Text #3

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2690. Min Act: -0.1700

Data Index: 2965721 (Open Web Text)

Max Activating Token Index: 123

Text #4

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2610. Min Act: -0.1700

Data Index: 1337426 (Open Web Text)

Max Activating Token Index: 539

Text #5

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2747. Min Act: -0.1700

Data Index: 3517017 (Open Web Text)

Max Activating Token Index: 1016

Text #6

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2193. Min Act: -0.1700

Data Index: 5995268 (Open Web Text)

Max Activating Token Index: 868

Text #7

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2125. Min Act: -0.1700

Data Index: 1858507 (Open Web Text)

Max Activating Token Index: 980

Text #8

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2331. Min Act: -0.1700

Data Index: 5607400 (Open Web Text)

Max Activating Token Index: 589

Text #9

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2340. Min Act: -0.1700

Data Index: 666815 (Open Web Text)

Max Activating Token Index: 365

Text #10

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2297. Min Act: -0.1700

Data Index: 5020210 (Open Web Text)

Max Activating Token Index: 697

Text #11

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2282. Min Act: -0.1700

Data Index: 8797412 (Open Web Text)

Max Activating Token Index: 637

Text #12

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2015. Min Act: -0.1700

Data Index: 5417080 (Open Web Text)

Max Activating Token Index: 66

Text #13

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.1903. Min Act: -0.1700

Data Index: 57888 (Open Web Text)

Max Activating Token Index: 758

Text #14

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.1829. Min Act: -0.1700

Data Index: 2416409 (Open Web Text)

Max Activating Token Index: 410

Text #15

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2089. Min Act: -0.1700

Data Index: 5364129 (Open Web Text)

Max Activating Token Index: 250

Text #16

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2136. Min Act: -0.1700

Data Index: 1977565 (Open Web Text)

Max Activating Token Index: 393

Text #17

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.1968. Min Act: -0.1700

Data Index: 4828457 (Open Web Text)

Max Activating Token Index: 112

Text #18

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2151. Min Act: -0.1700

Data Index: 2486982 (Open Web Text)

Max Activating Token Index: 776

Text #19

Max Range: 4.3991. Min Range: -4.3991

Max Act: 4.2083. Min Act: -0.1700

Data Index: 4474920 (Open Web Text)

Max Activating Token Index: 6