Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1298 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.7424. Min Act: -0.1700

Data Index: 5011541 (Open Web Text)

Max Activating Token Index: 765

Text #1

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.7021. Min Act: -0.1700

Data Index: 6122922 (Open Web Text)

Max Activating Token Index: 182

Text #2

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.6911. Min Act: -0.1700

Data Index: 6535200 (Open Web Text)

Max Activating Token Index: 461

Text #3

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.6894. Min Act: -0.1700

Data Index: 6387142 (Open Web Text)

Max Activating Token Index: 372

Text #4

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.6494. Min Act: -0.1700

Data Index: 3590911 (Open Web Text)

Max Activating Token Index: 655

Text #5

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.6042. Min Act: -0.1700

Data Index: 6502165 (Open Web Text)

Max Activating Token Index: 815

Text #6

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5874. Min Act: -0.1700

Data Index: 1344598 (Open Web Text)

Max Activating Token Index: 417

Text #7

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5685. Min Act: -0.1700

Data Index: 6122193 (Open Web Text)

Max Activating Token Index: 702

Text #8

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5768. Min Act: -0.1700

Data Index: 3177343 (Open Web Text)

Max Activating Token Index: 888

Text #9

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5432. Min Act: -0.1700

Data Index: 8759112 (Open Web Text)

Max Activating Token Index: 654

Text #10

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5273. Min Act: -0.1700

Data Index: 5844481 (Open Web Text)

Max Activating Token Index: 24

Text #11

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5032. Min Act: -0.1700

Data Index: 395602 (Open Web Text)

Max Activating Token Index: 248

Text #12

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4913. Min Act: -0.1700

Data Index: 8342976 (Open Web Text)

Max Activating Token Index: 152

Text #13

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.5298. Min Act: -0.1700

Data Index: 1168598 (Open Web Text)

Max Activating Token Index: 762

Text #14

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4963. Min Act: -0.1700

Data Index: 299778 (Open Web Text)

Max Activating Token Index: 192

Text #15

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4808. Min Act: -0.1700

Data Index: 995467 (Open Web Text)

Max Activating Token Index: 918

Text #16

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4955. Min Act: -0.1700

Data Index: 8152502 (Open Web Text)

Max Activating Token Index: 176

Text #17

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4766. Min Act: -0.1700

Data Index: 5634560 (Open Web Text)

Max Activating Token Index: 148

Text #18

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4657. Min Act: -0.1700

Data Index: 1974844 (Open Web Text)

Max Activating Token Index: 777

Text #19

Max Range: 3.7424. Min Range: -3.7424

Max Act: 3.4589. Min Act: -0.1700

Data Index: 7844511 (Open Web Text)

Max Activating Token Index: 171