Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 649 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.4317. Min Act: -0.1700

Data Index: 1798480 (Open Web Text)

Max Activating Token Index: 90

Text #1

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.4568. Min Act: -0.1700

Data Index: 6998138 (Open Web Text)

Max Activating Token Index: 48

Text #2

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.4109. Min Act: -0.1700

Data Index: 6698138 (Open Web Text)

Max Activating Token Index: 189

Text #3

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3912. Min Act: -0.1700

Data Index: 3202861 (Open Web Text)

Max Activating Token Index: 131

Text #4

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.4003. Min Act: -0.1700

Data Index: 5385225 (Open Web Text)

Max Activating Token Index: 343

Text #5

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3786. Min Act: -0.1700

Data Index: 8363858 (Open Web Text)

Max Activating Token Index: 165

Text #6

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3901. Min Act: -0.1700

Data Index: 6122343 (Open Web Text)

Max Activating Token Index: 1003

Text #7

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3580. Min Act: -0.1700

Data Index: 4838689 (Open Web Text)

Max Activating Token Index: 840

Text #8

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3714. Min Act: -0.1700

Data Index: 2352913 (Open Web Text)

Max Activating Token Index: 354

Text #9

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3653. Min Act: -0.1700

Data Index: 6610629 (Open Web Text)

Max Activating Token Index: 110

Text #10

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3449. Min Act: -0.1700

Data Index: 1543773 (Open Web Text)

Max Activating Token Index: 186

Text #11

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3475. Min Act: -0.1700

Data Index: 4553240 (Open Web Text)

Max Activating Token Index: 270

Text #12

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3266. Min Act: -0.1700

Data Index: 2283649 (Open Web Text)

Max Activating Token Index: 102

Text #13

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3413. Min Act: -0.1700

Data Index: 1815952 (Open Web Text)

Max Activating Token Index: 322

Text #14

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3188. Min Act: -0.1700

Data Index: 971614 (Open Web Text)

Max Activating Token Index: 422

Text #15

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3248. Min Act: -0.1700

Data Index: 1817956 (Open Web Text)

Max Activating Token Index: 45

Text #16

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3175. Min Act: -0.1700

Data Index: 1414971 (Open Web Text)

Max Activating Token Index: 209

Text #17

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3289. Min Act: -0.1700

Data Index: 694829 (Open Web Text)

Max Activating Token Index: 495

Text #18

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3173. Min Act: -0.1700

Data Index: 4067957 (Open Web Text)

Max Activating Token Index: 209

Text #19

Max Range: 4.4568. Min Range: -4.4568

Max Act: 4.3067. Min Act: -0.1700

Data Index: 42436 (Open Web Text)

Max Activating Token Index: 782