Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2181 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.3080. Min Act: -0.1700

Data Index: 2680657 (Open Web Text)

Max Activating Token Index: 641

Text #1

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2623. Min Act: -0.1700

Data Index: 3957855 (Open Web Text)

Max Activating Token Index: 243

Text #2

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2650. Min Act: -0.1700

Data Index: 3148130 (Open Web Text)

Max Activating Token Index: 376

Text #3

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2438. Min Act: -0.1700

Data Index: 4277488 (Open Web Text)

Max Activating Token Index: 169

Text #4

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2353. Min Act: -0.1700

Data Index: 8427114 (Open Web Text)

Max Activating Token Index: 355

Text #5

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2387. Min Act: -0.1700

Data Index: 6464199 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2093. Min Act: -0.1700

Data Index: 6614962 (Open Web Text)

Max Activating Token Index: 251

Text #7

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2169. Min Act: -0.1700

Data Index: 2891735 (Open Web Text)

Max Activating Token Index: 881

Text #8

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2172. Min Act: -0.1700

Data Index: 8534453 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2151. Min Act: -0.1700

Data Index: 4168150 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2029. Min Act: -0.1700

Data Index: 4437982 (Open Web Text)

Max Activating Token Index: 892

Text #11

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.2100. Min Act: -0.1700

Data Index: 7162452 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1962. Min Act: -0.1700

Data Index: 3507259 (Open Web Text)

Max Activating Token Index: 871

Text #13

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1870. Min Act: -0.1700

Data Index: 991729 (Open Web Text)

Max Activating Token Index: 157

Text #14

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1797. Min Act: -0.1700

Data Index: 4717088 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1763. Min Act: -0.1700

Data Index: 5813190 (Open Web Text)

Max Activating Token Index: 938

Text #16

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1812. Min Act: -0.1700

Data Index: 8102770 (Open Web Text)

Max Activating Token Index: 245

Text #17

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1761. Min Act: -0.1700

Data Index: 3590693 (Open Web Text)

Max Activating Token Index: 679

Text #18

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1584. Min Act: -0.1700

Data Index: 1616381 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 2.3080. Min Range: -2.3080

Max Act: 2.1678. Min Act: -0.1700

Data Index: 2231796 (Open Web Text)

Max Activating Token Index: 134