Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 488 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1532. Min Range: -4.1532

Max Act: 4.1532. Min Act: -0.1700

Data Index: 1937386 (Open Web Text)

Max Activating Token Index: 768

Text #1

Max Range: 4.1532. Min Range: -4.1532

Max Act: 4.0364. Min Act: -0.1700

Data Index: 1685353 (Open Web Text)

Max Activating Token Index: 913

Text #2

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.9164. Min Act: -0.1700

Data Index: 7737004 (Open Web Text)

Max Activating Token Index: 412

Text #3

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8993. Min Act: -0.1700

Data Index: 314882 (Open Web Text)

Max Activating Token Index: 291

Text #4

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.9108. Min Act: -0.1700

Data Index: 7705217 (Open Web Text)

Max Activating Token Index: 722

Text #5

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8338. Min Act: -0.1700

Data Index: 3597115 (Open Web Text)

Max Activating Token Index: 350

Text #6

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8176. Min Act: -0.1700

Data Index: 6618363 (Open Web Text)

Max Activating Token Index: 512

Text #7

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8720. Min Act: -0.1700

Data Index: 7592086 (Open Web Text)

Max Activating Token Index: 254

Text #8

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8058. Min Act: -0.1700

Data Index: 1851768 (Open Web Text)

Max Activating Token Index: 569

Text #9

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8219. Min Act: -0.1700

Data Index: 2956326 (Open Web Text)

Max Activating Token Index: 372

Text #10

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8044. Min Act: -0.1700

Data Index: 729023 (Open Web Text)

Max Activating Token Index: 976

Text #11

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8248. Min Act: -0.1700

Data Index: 7781674 (Open Web Text)

Max Activating Token Index: 684

Text #12

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.7911. Min Act: -0.1700

Data Index: 2960840 (Open Web Text)

Max Activating Token Index: 978

Text #13

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8369. Min Act: -0.1700

Data Index: 7477523 (Open Web Text)

Max Activating Token Index: 231

Text #14

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.7952. Min Act: -0.1700

Data Index: 5477534 (Open Web Text)

Max Activating Token Index: 938

Text #15

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8153. Min Act: -0.1700

Data Index: 5071958 (Open Web Text)

Max Activating Token Index: 445

Text #16

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.8031. Min Act: -0.1700

Data Index: 3387338 (Open Web Text)

Max Activating Token Index: 236

Text #17

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.7779. Min Act: -0.1700

Data Index: 254233 (Open Web Text)

Max Activating Token Index: 291

Text #18

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.7918. Min Act: -0.1700

Data Index: 4900722 (Open Web Text)

Max Activating Token Index: 957

Text #19

Max Range: 4.1532. Min Range: -4.1532

Max Act: 3.7901. Min Act: -0.1700

Data Index: 8646848 (Open Web Text)

Max Activating Token Index: 299