Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1957 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.7824. Min Act: -0.1700

Data Index: 7234728 (Open Web Text)

Max Activating Token Index: 918

Text #1

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.7638. Min Act: -0.1700

Data Index: 3897636 (Open Web Text)

Max Activating Token Index: 263

Text #2

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6398. Min Act: -0.1700

Data Index: 8161807 (Open Web Text)

Max Activating Token Index: 397

Text #3

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6194. Min Act: -0.1700

Data Index: 1712110 (Open Web Text)

Max Activating Token Index: 209

Text #4

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6360. Min Act: -0.1700

Data Index: 5064803 (Open Web Text)

Max Activating Token Index: 766

Text #5

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6227. Min Act: -0.1700

Data Index: 5506019 (Open Web Text)

Max Activating Token Index: 482

Text #6

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6284. Min Act: -0.1700

Data Index: 3081352 (Open Web Text)

Max Activating Token Index: 254

Text #7

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5911. Min Act: -0.1700

Data Index: 3761047 (Open Web Text)

Max Activating Token Index: 921

Text #8

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6001. Min Act: -0.1700

Data Index: 7324664 (Open Web Text)

Max Activating Token Index: 17

Text #9

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5617. Min Act: -0.1700

Data Index: 2966121 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.6101. Min Act: -0.1700

Data Index: 7218792 (Open Web Text)

Max Activating Token Index: 80

Text #11

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5908. Min Act: -0.1700

Data Index: 1620726 (Open Web Text)

Max Activating Token Index: 860

Text #12

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5954. Min Act: -0.1700

Data Index: 6364347 (Open Web Text)

Max Activating Token Index: 26

Text #13

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5982. Min Act: -0.1700

Data Index: 6129992 (Open Web Text)

Max Activating Token Index: 816

Text #14

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5824. Min Act: -0.1700

Data Index: 4355241 (Open Web Text)

Max Activating Token Index: 845

Text #15

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5823. Min Act: -0.1700

Data Index: 6423861 (Open Web Text)

Max Activating Token Index: 734

Text #16

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5806. Min Act: -0.1700

Data Index: 1369326 (Open Web Text)

Max Activating Token Index: 294

Text #17

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5766. Min Act: -0.1700

Data Index: 3520433 (Open Web Text)

Max Activating Token Index: 316

Text #18

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5555. Min Act: -0.1700

Data Index: 580394 (Open Web Text)

Max Activating Token Index: 17

Text #19

Max Range: 3.7824. Min Range: -3.7824

Max Act: 3.5368. Min Act: -0.1700

Data Index: 2239523 (Open Web Text)

Max Activating Token Index: 5