Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 768 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.2853. Min Act: -0.1700

Data Index: 1508828 (Open Web Text)

Max Activating Token Index: 376

Text #1

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.2560. Min Act: -0.1700

Data Index: 7139515 (Open Web Text)

Max Activating Token Index: 394

Text #2

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.2189. Min Act: -0.1700

Data Index: 4705687 (Open Web Text)

Max Activating Token Index: 341

Text #3

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.1555. Min Act: -0.1700

Data Index: 1480608 (Open Web Text)

Max Activating Token Index: 888

Text #4

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0759. Min Act: -0.1700

Data Index: 6840648 (Open Web Text)

Max Activating Token Index: 721

Text #5

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0905. Min Act: -0.1700

Data Index: 1254393 (Open Web Text)

Max Activating Token Index: 663

Text #6

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0485. Min Act: -0.1700

Data Index: 155348 (Open Web Text)

Max Activating Token Index: 70

Text #7

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0650. Min Act: -0.1700

Data Index: 4241875 (Open Web Text)

Max Activating Token Index: 148

Text #8

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0531. Min Act: -0.1700

Data Index: 3656651 (Open Web Text)

Max Activating Token Index: 694

Text #9

Max Range: 5.2853. Min Range: -5.2853

Max Act: 4.9988. Min Act: -0.1700

Data Index: 3803116 (Open Web Text)

Max Activating Token Index: 454

Text #10

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0207. Min Act: -0.1700

Data Index: 8677042 (Open Web Text)

Max Activating Token Index: 483

Text #11

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0317. Min Act: -0.1700

Data Index: 2123316 (Open Web Text)

Max Activating Token Index: 653

Text #12

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0232. Min Act: -0.1700

Data Index: 8470718 (Open Web Text)

Max Activating Token Index: 874

Text #13

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0428. Min Act: -0.1700

Data Index: 4781041 (Open Web Text)

Max Activating Token Index: 898

Text #14

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0099. Min Act: -0.1700

Data Index: 2056371 (Open Web Text)

Max Activating Token Index: 764

Text #15

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0355. Min Act: -0.1700

Data Index: 5127233 (Open Web Text)

Max Activating Token Index: 333

Text #16

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0210. Min Act: -0.1700

Data Index: 5320121 (Open Web Text)

Max Activating Token Index: 141

Text #17

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0138. Min Act: -0.1700

Data Index: 206245 (Open Web Text)

Max Activating Token Index: 452

Text #18

Max Range: 5.2853. Min Range: -5.2853

Max Act: 5.0127. Min Act: -0.1700

Data Index: 3064517 (Open Web Text)

Max Activating Token Index: 798

Text #19

Max Range: 5.2853. Min Range: -5.2853

Max Act: 4.9881. Min Act: -0.1700

Data Index: 4977066 (Open Web Text)

Max Activating Token Index: 229