Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 449 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.3151. Min Act: -0.1700

Data Index: 4668754 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.2563. Min Act: -0.1700

Data Index: 8108240 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.2042. Min Act: -0.1700

Data Index: 7980087 (Open Web Text)

Max Activating Token Index: 372

Text #3

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1960. Min Act: -0.1700

Data Index: 7980086 (Open Web Text)

Max Activating Token Index: 504

Text #4

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1906. Min Act: -0.1700

Data Index: 4295232 (Open Web Text)

Max Activating Token Index: 643

Text #5

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1873. Min Act: -0.1700

Data Index: 7980091 (Open Web Text)

Max Activating Token Index: 522

Text #6

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1943. Min Act: -0.1700

Data Index: 7980095 (Open Web Text)

Max Activating Token Index: 718

Text #7

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1944. Min Act: -0.1700

Data Index: 7980090 (Open Web Text)

Max Activating Token Index: 672

Text #8

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1825. Min Act: -0.1700

Data Index: 2048966 (Open Web Text)

Max Activating Token Index: 854

Text #9

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1835. Min Act: -0.1700

Data Index: 3891738 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1699. Min Act: -0.1700

Data Index: 7385104 (Open Web Text)

Max Activating Token Index: 621

Text #11

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1724. Min Act: -0.1700

Data Index: 4220337 (Open Web Text)

Max Activating Token Index: 310

Text #12

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1815. Min Act: -0.1700

Data Index: 7980098 (Open Web Text)

Max Activating Token Index: 229

Text #13

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1790. Min Act: -0.1700

Data Index: 7980097 (Open Web Text)

Max Activating Token Index: 436

Text #14

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1781. Min Act: -0.1700

Data Index: 4431145 (Open Web Text)

Max Activating Token Index: 160

Text #15

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1763. Min Act: -0.1700

Data Index: 7980092 (Open Web Text)

Max Activating Token Index: 390

Text #16

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1567. Min Act: -0.1700

Data Index: 7213479 (Open Web Text)

Max Activating Token Index: 13

Text #17

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1678. Min Act: -0.1700

Data Index: 7232197 (Open Web Text)

Max Activating Token Index: 937

Text #18

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1559. Min Act: -0.1700

Data Index: 7050106 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 2.3151. Min Range: -2.3151

Max Act: 2.1643. Min Act: -0.1700

Data Index: 2502671 (Open Web Text)

Max Activating Token Index: 346