Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1756 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0814. Min Range: -5.0814

Max Act: 5.0814. Min Act: -0.1700

Data Index: 8356658 (Open Web Text)

Max Activating Token Index: 518

Text #1

Max Range: 5.0814. Min Range: -5.0814

Max Act: 5.0136. Min Act: -0.1700

Data Index: 5314778 (Open Web Text)

Max Activating Token Index: 563

Text #2

Max Range: 5.0814. Min Range: -5.0814

Max Act: 5.0328. Min Act: -0.1700

Data Index: 4736216 (Open Web Text)

Max Activating Token Index: 297

Text #3

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.9541. Min Act: -0.1700

Data Index: 5223951 (Open Web Text)

Max Activating Token Index: 438

Text #4

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.9448. Min Act: -0.1700

Data Index: 1534025 (Open Web Text)

Max Activating Token Index: 560

Text #5

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.9169. Min Act: -0.1700

Data Index: 8361883 (Open Web Text)

Max Activating Token Index: 227

Text #6

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8276. Min Act: -0.1700

Data Index: 7469227 (Open Web Text)

Max Activating Token Index: 795

Text #7

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8619. Min Act: -0.1700

Data Index: 209144 (Open Web Text)

Max Activating Token Index: 195

Text #8

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8345. Min Act: -0.1700

Data Index: 8196466 (Open Web Text)

Max Activating Token Index: 661

Text #9

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8467. Min Act: -0.1700

Data Index: 8630658 (Open Web Text)

Max Activating Token Index: 221

Text #10

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8530. Min Act: -0.1700

Data Index: 2283158 (Open Web Text)

Max Activating Token Index: 594

Text #11

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7982. Min Act: -0.1700

Data Index: 1444813 (Open Web Text)

Max Activating Token Index: 132

Text #12

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8087. Min Act: -0.1700

Data Index: 4463554 (Open Web Text)

Max Activating Token Index: 28

Text #13

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7930. Min Act: -0.1700

Data Index: 955990 (Open Web Text)

Max Activating Token Index: 49

Text #14

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8136. Min Act: -0.1700

Data Index: 7967416 (Open Web Text)

Max Activating Token Index: 940

Text #15

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7679. Min Act: -0.1700

Data Index: 7656522 (Open Web Text)

Max Activating Token Index: 297

Text #16

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.8275. Min Act: -0.1700

Data Index: 4914667 (Open Web Text)

Max Activating Token Index: 1008

Text #17

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7701. Min Act: -0.1700

Data Index: 3283122 (Open Web Text)

Max Activating Token Index: 608

Text #18

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7843. Min Act: -0.1700

Data Index: 6428250 (Open Web Text)

Max Activating Token Index: 111

Text #19

Max Range: 5.0814. Min Range: -5.0814

Max Act: 4.7256. Min Act: -0.1700

Data Index: 7901148 (Open Web Text)

Max Activating Token Index: 319