Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1286 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.6543. Min Act: -0.1700

Data Index: 4589067 (Open Web Text)

Max Activating Token Index: 40

Text #1

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.6655. Min Act: -0.1700

Data Index: 3449518 (Open Web Text)

Max Activating Token Index: 374

Text #2

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.6120. Min Act: -0.1700

Data Index: 390137 (Open Web Text)

Max Activating Token Index: 306

Text #3

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.6207. Min Act: -0.1700

Data Index: 2019317 (Open Web Text)

Max Activating Token Index: 226

Text #4

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.6140. Min Act: -0.1700

Data Index: 5274592 (Open Web Text)

Max Activating Token Index: 168

Text #5

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5541. Min Act: -0.1700

Data Index: 4584712 (Open Web Text)

Max Activating Token Index: 875

Text #6

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5599. Min Act: -0.1700

Data Index: 1609903 (Open Web Text)

Max Activating Token Index: 674

Text #7

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5736. Min Act: -0.1700

Data Index: 1661265 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5433. Min Act: -0.1700

Data Index: 2695665 (Open Web Text)

Max Activating Token Index: 198

Text #9

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5105. Min Act: -0.1700

Data Index: 5816478 (Open Web Text)

Max Activating Token Index: 708

Text #10

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5058. Min Act: -0.1700

Data Index: 5748107 (Open Web Text)

Max Activating Token Index: 46

Text #11

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5331. Min Act: -0.1700

Data Index: 5556169 (Open Web Text)

Max Activating Token Index: 444

Text #12

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5242. Min Act: -0.1700

Data Index: 3387897 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5234. Min Act: -0.1700

Data Index: 7651479 (Open Web Text)

Max Activating Token Index: 615

Text #14

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5190. Min Act: -0.1700

Data Index: 1403669 (Open Web Text)

Max Activating Token Index: 598

Text #15

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5062. Min Act: -0.1700

Data Index: 3523718 (Open Web Text)

Max Activating Token Index: 409

Text #16

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5076. Min Act: -0.1700

Data Index: 7156237 (Open Web Text)

Max Activating Token Index: 834

Text #17

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5010. Min Act: -0.1700

Data Index: 4156079 (Open Web Text)

Max Activating Token Index: 7

Text #18

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.4899. Min Act: -0.1700

Data Index: 7473648 (Open Web Text)

Max Activating Token Index: 365

Text #19

Max Range: 3.6655. Min Range: -3.6655

Max Act: 3.5037. Min Act: -0.1700

Data Index: 7495878 (Open Web Text)

Max Activating Token Index: 146