Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1434 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.3938. Min Act: -0.1700

Data Index: 7922361 (Open Web Text)

Max Activating Token Index: 453

Text #1

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.3287. Min Act: -0.1700

Data Index: 888540 (Open Web Text)

Max Activating Token Index: 87

Text #2

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.2460. Min Act: -0.1700

Data Index: 8323814 (Open Web Text)

Max Activating Token Index: 454

Text #3

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.2315. Min Act: -0.1700

Data Index: 6059710 (Open Web Text)

Max Activating Token Index: 23

Text #4

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.2146. Min Act: -0.1700

Data Index: 7527960 (Open Web Text)

Max Activating Token Index: 518

Text #5

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1799. Min Act: -0.1700

Data Index: 2977447 (Open Web Text)

Max Activating Token Index: 293

Text #6

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1559. Min Act: -0.1700

Data Index: 6053978 (Open Web Text)

Max Activating Token Index: 1012

Text #7

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1955. Min Act: -0.1700

Data Index: 7017606 (Open Web Text)

Max Activating Token Index: 97

Text #8

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1656. Min Act: -0.1700

Data Index: 5265697 (Open Web Text)

Max Activating Token Index: 896

Text #9

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1617. Min Act: -0.1700

Data Index: 2046502 (Open Web Text)

Max Activating Token Index: 146

Text #10

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1540. Min Act: -0.1700

Data Index: 1987085 (Open Web Text)

Max Activating Token Index: 789

Text #11

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1098. Min Act: -0.1700

Data Index: 4370828 (Open Web Text)

Max Activating Token Index: 290

Text #12

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1221. Min Act: -0.1700

Data Index: 4374962 (Open Web Text)

Max Activating Token Index: 296

Text #13

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1230. Min Act: -0.1700

Data Index: 7527963 (Open Web Text)

Max Activating Token Index: 661

Text #14

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.0898. Min Act: -0.1700

Data Index: 3381528 (Open Web Text)

Max Activating Token Index: 137

Text #15

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.1013. Min Act: -0.1700

Data Index: 7232550 (Open Web Text)

Max Activating Token Index: 740

Text #16

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.0714. Min Act: -0.1700

Data Index: 4700260 (Open Web Text)

Max Activating Token Index: 738

Text #17

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.0883. Min Act: -0.1700

Data Index: 6906158 (Open Web Text)

Max Activating Token Index: 20

Text #18

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.0689. Min Act: -0.1700

Data Index: 5211682 (Open Web Text)

Max Activating Token Index: 555

Text #19

Max Range: 3.3938. Min Range: -3.3938

Max Act: 3.0720. Min Act: -0.1700

Data Index: 182343 (Open Web Text)

Max Activating Token Index: 355