Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 228 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.5284. Min Act: -0.1700

Data Index: 4714461 (Open Web Text)

Max Activating Token Index: 821

Text #1

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.5188. Min Act: -0.1700

Data Index: 4258996 (Open Web Text)

Max Activating Token Index: 85

Text #2

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.4847. Min Act: -0.1700

Data Index: 8268225 (Open Web Text)

Max Activating Token Index: 734

Text #3

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.4449. Min Act: -0.1700

Data Index: 662875 (Open Web Text)

Max Activating Token Index: 146

Text #4

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3902. Min Act: -0.1700

Data Index: 2477999 (Open Web Text)

Max Activating Token Index: 178

Text #5

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3834. Min Act: -0.1700

Data Index: 1313995 (Open Web Text)

Max Activating Token Index: 161

Text #6

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.4389. Min Act: -0.1700

Data Index: 443238 (Open Web Text)

Max Activating Token Index: 511

Text #7

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3786. Min Act: -0.1700

Data Index: 5753133 (Open Web Text)

Max Activating Token Index: 182

Text #8

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.4201. Min Act: -0.1700

Data Index: 5967903 (Open Web Text)

Max Activating Token Index: 762

Text #9

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3626. Min Act: -0.1700

Data Index: 8105861 (Open Web Text)

Max Activating Token Index: 411

Text #10

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3119. Min Act: -0.1700

Data Index: 7489874 (Open Web Text)

Max Activating Token Index: 659

Text #11

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.4020. Min Act: -0.1700

Data Index: 4317678 (Open Web Text)

Max Activating Token Index: 990

Text #12

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3580. Min Act: -0.1700

Data Index: 8749599 (Open Web Text)

Max Activating Token Index: 480

Text #13

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3190. Min Act: -0.1700

Data Index: 3848345 (Open Web Text)

Max Activating Token Index: 524

Text #14

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3330. Min Act: -0.1700

Data Index: 7185992 (Open Web Text)

Max Activating Token Index: 504

Text #15

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3217. Min Act: -0.1700

Data Index: 5072142 (Open Web Text)

Max Activating Token Index: 487

Text #16

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3065. Min Act: -0.1700

Data Index: 1207747 (Open Web Text)

Max Activating Token Index: 135

Text #17

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.2959. Min Act: -0.1700

Data Index: 1625731 (Open Web Text)

Max Activating Token Index: 291

Text #18

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.3289. Min Act: -0.1700

Data Index: 3412635 (Open Web Text)

Max Activating Token Index: 678

Text #19

Max Range: 4.5284. Min Range: -4.5284

Max Act: 4.2996. Min Act: -0.1700

Data Index: 1228211 (Open Web Text)

Max Activating Token Index: 417