Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 724 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.8123. Min Act: -0.1700

Data Index: 3201094 (Open Web Text)

Max Activating Token Index: 157

Text #1

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.8175. Min Act: -0.1700

Data Index: 1320125 (Open Web Text)

Max Activating Token Index: 377

Text #2

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.8242. Min Act: -0.1700

Data Index: 2136311 (Open Web Text)

Max Activating Token Index: 649

Text #3

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7984. Min Act: -0.1700

Data Index: 1440240 (Open Web Text)

Max Activating Token Index: 698

Text #4

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7671. Min Act: -0.1700

Data Index: 6795400 (Open Web Text)

Max Activating Token Index: 364

Text #5

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7709. Min Act: -0.1700

Data Index: 2655264 (Open Web Text)

Max Activating Token Index: 212

Text #6

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7965. Min Act: -0.1700

Data Index: 2628911 (Open Web Text)

Max Activating Token Index: 271

Text #7

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7835. Min Act: -0.1700

Data Index: 2498282 (Open Web Text)

Max Activating Token Index: 337

Text #8

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7721. Min Act: -0.1700

Data Index: 1577099 (Open Web Text)

Max Activating Token Index: 114

Text #9

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7777. Min Act: -0.1700

Data Index: 1562553 (Open Web Text)

Max Activating Token Index: 243

Text #10

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7889. Min Act: -0.1700

Data Index: 6662491 (Open Web Text)

Max Activating Token Index: 354

Text #11

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7921. Min Act: -0.1700

Data Index: 5428316 (Open Web Text)

Max Activating Token Index: 112

Text #12

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7771. Min Act: -0.1700

Data Index: 1130299 (Open Web Text)

Max Activating Token Index: 454

Text #13

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7705. Min Act: -0.1700

Data Index: 3925416 (Open Web Text)

Max Activating Token Index: 373

Text #14

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7351. Min Act: -0.1700

Data Index: 2183150 (Open Web Text)

Max Activating Token Index: 223

Text #15

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7674. Min Act: -0.1700

Data Index: 1999311 (Open Web Text)

Max Activating Token Index: 416

Text #16

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7583. Min Act: -0.1700

Data Index: 592262 (Open Web Text)

Max Activating Token Index: 718

Text #17

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7635. Min Act: -0.1700

Data Index: 3716893 (Open Web Text)

Max Activating Token Index: 673

Text #18

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7312. Min Act: -0.1700

Data Index: 3232419 (Open Web Text)

Max Activating Token Index: 201

Text #19

Max Range: 4.8242. Min Range: -4.8242

Max Act: 4.7228. Min Act: -0.1700

Data Index: 3412363 (Open Web Text)

Max Activating Token Index: 776