Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 900 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3469. Min Act: -0.1700

Data Index: 1116303 (Open Web Text)

Max Activating Token Index: 1002

Text #1

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3592. Min Act: -0.1700

Data Index: 8003435 (Open Web Text)

Max Activating Token Index: 522

Text #2

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3445. Min Act: -0.1700

Data Index: 2657690 (Open Web Text)

Max Activating Token Index: 363

Text #3

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3307. Min Act: -0.1700

Data Index: 6816289 (Open Web Text)

Max Activating Token Index: 374

Text #4

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3289. Min Act: -0.1700

Data Index: 4207591 (Open Web Text)

Max Activating Token Index: 927

Text #5

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2995. Min Act: -0.1700

Data Index: 3020199 (Open Web Text)

Max Activating Token Index: 605

Text #6

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2986. Min Act: -0.1700

Data Index: 8461812 (Open Web Text)

Max Activating Token Index: 201

Text #7

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3103. Min Act: -0.1700

Data Index: 7983388 (Open Web Text)

Max Activating Token Index: 327

Text #8

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.3021. Min Act: -0.1700

Data Index: 5758238 (Open Web Text)

Max Activating Token Index: 428

Text #9

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2885. Min Act: -0.1700

Data Index: 8133317 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2637. Min Act: -0.1700

Data Index: 3284337 (Open Web Text)

Max Activating Token Index: 636

Text #11

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2838. Min Act: -0.1700

Data Index: 4633614 (Open Web Text)

Max Activating Token Index: 500

Text #12

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2723. Min Act: -0.1700

Data Index: 6934389 (Open Web Text)

Max Activating Token Index: 794

Text #13

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2575. Min Act: -0.1700

Data Index: 772980 (Open Web Text)

Max Activating Token Index: 140

Text #14

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2466. Min Act: -0.1700

Data Index: 7271981 (Open Web Text)

Max Activating Token Index: 319

Text #15

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2250. Min Act: -0.1700

Data Index: 8039864 (Open Web Text)

Max Activating Token Index: 453

Text #16

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2453. Min Act: -0.1700

Data Index: 2830371 (Open Web Text)

Max Activating Token Index: 371

Text #17

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2610. Min Act: -0.1700

Data Index: 3485030 (Open Web Text)

Max Activating Token Index: 446

Text #18

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2394. Min Act: -0.1700

Data Index: 8343354 (Open Web Text)

Max Activating Token Index: 222

Text #19

Max Range: 3.3592. Min Range: -3.3592

Max Act: 3.2183. Min Act: -0.1700

Data Index: 6178811 (Open Web Text)

Max Activating Token Index: 97