Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2163 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1006. Min Range: -3.1006

Max Act: 3.1006. Min Act: -0.1700

Data Index: 6657871 (Open Web Text)

Max Activating Token Index: 797

Text #1

Max Range: 3.1006. Min Range: -3.1006

Max Act: 3.0018. Min Act: -0.1700

Data Index: 5509337 (Open Web Text)

Max Activating Token Index: 240

Text #2

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9870. Min Act: -0.1700

Data Index: 1183456 (Open Web Text)

Max Activating Token Index: 47

Text #3

Max Range: 3.1006. Min Range: -3.1006

Max Act: 3.0020. Min Act: -0.1700

Data Index: 4881307 (Open Web Text)

Max Activating Token Index: 916

Text #4

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9875. Min Act: -0.1700

Data Index: 6662113 (Open Web Text)

Max Activating Token Index: 907

Text #5

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9799. Min Act: -0.1700

Data Index: 2791399 (Open Web Text)

Max Activating Token Index: 129

Text #6

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9560. Min Act: -0.1700

Data Index: 6633064 (Open Web Text)

Max Activating Token Index: 914

Text #7

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9582. Min Act: -0.1700

Data Index: 1643903 (Open Web Text)

Max Activating Token Index: 190

Text #8

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9301. Min Act: -0.1700

Data Index: 8120302 (Open Web Text)

Max Activating Token Index: 426

Text #9

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9038. Min Act: -0.1700

Data Index: 5900983 (Open Web Text)

Max Activating Token Index: 892

Text #10

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9033. Min Act: -0.1700

Data Index: 6637046 (Open Web Text)

Max Activating Token Index: 618

Text #11

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9319. Min Act: -0.1700

Data Index: 6437077 (Open Web Text)

Max Activating Token Index: 995

Text #12

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.9204. Min Act: -0.1700

Data Index: 2556365 (Open Web Text)

Max Activating Token Index: 791

Text #13

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8953. Min Act: -0.1700

Data Index: 3647301 (Open Web Text)

Max Activating Token Index: 359

Text #14

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8817. Min Act: -0.1700

Data Index: 7212681 (Open Web Text)

Max Activating Token Index: 409

Text #15

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8958. Min Act: -0.1700

Data Index: 3701128 (Open Web Text)

Max Activating Token Index: 654

Text #16

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8881. Min Act: -0.1700

Data Index: 7173704 (Open Web Text)

Max Activating Token Index: 921

Text #17

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8844. Min Act: -0.1700

Data Index: 808095 (Open Web Text)

Max Activating Token Index: 946

Text #18

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8602. Min Act: -0.1700

Data Index: 289322 (Open Web Text)

Max Activating Token Index: 751

Text #19

Max Range: 3.1006. Min Range: -3.1006

Max Act: 2.8717. Min Act: -0.1700

Data Index: 3556172 (Open Web Text)

Max Activating Token Index: 375