Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1045 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.7171. Min Act: -0.1700

Data Index: 1247174 (Open Web Text)

Max Activating Token Index: 683

Text #1

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6896. Min Act: -0.1700

Data Index: 6986577 (Open Web Text)

Max Activating Token Index: 248

Text #2

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6865. Min Act: -0.1700

Data Index: 1247173 (Open Web Text)

Max Activating Token Index: 284

Text #3

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6706. Min Act: -0.1700

Data Index: 8701763 (Open Web Text)

Max Activating Token Index: 345

Text #4

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6384. Min Act: -0.1700

Data Index: 5233307 (Open Web Text)

Max Activating Token Index: 961

Text #5

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6414. Min Act: -0.1700

Data Index: 506595 (Open Web Text)

Max Activating Token Index: 302

Text #6

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6336. Min Act: -0.1700

Data Index: 8702840 (Open Web Text)

Max Activating Token Index: 531

Text #7

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6279. Min Act: -0.1700

Data Index: 3239532 (Open Web Text)

Max Activating Token Index: 115

Text #8

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6248. Min Act: -0.1700

Data Index: 6145492 (Open Web Text)

Max Activating Token Index: 105

Text #9

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6302. Min Act: -0.1700

Data Index: 6892663 (Open Web Text)

Max Activating Token Index: 89

Text #10

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6300. Min Act: -0.1700

Data Index: 1251127 (Open Web Text)

Max Activating Token Index: 47

Text #11

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6315. Min Act: -0.1700

Data Index: 3616811 (Open Web Text)

Max Activating Token Index: 44

Text #12

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6229. Min Act: -0.1700

Data Index: 4876881 (Open Web Text)

Max Activating Token Index: 575

Text #13

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6117. Min Act: -0.1700

Data Index: 6124547 (Open Web Text)

Max Activating Token Index: 70

Text #14

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6140. Min Act: -0.1700

Data Index: 4932074 (Open Web Text)

Max Activating Token Index: 689

Text #15

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6146. Min Act: -0.1700

Data Index: 4589859 (Open Web Text)

Max Activating Token Index: 148

Text #16

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6069. Min Act: -0.1700

Data Index: 6696157 (Open Web Text)

Max Activating Token Index: 951

Text #17

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6031. Min Act: -0.1700

Data Index: 7446414 (Open Web Text)

Max Activating Token Index: 204

Text #18

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.6011. Min Act: -0.1700

Data Index: 1637809 (Open Web Text)

Max Activating Token Index: 211

Text #19

Max Range: 1.7171. Min Range: -1.7171

Max Act: 1.5936. Min Act: -0.1700

Data Index: 2443827 (Open Web Text)

Max Activating Token Index: 58