Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1202 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.4406. Min Act: -0.1700

Data Index: 3667600 (Open Web Text)

Max Activating Token Index: 180

Text #1

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.3727. Min Act: -0.1700

Data Index: 6647205 (Open Web Text)

Max Activating Token Index: 36

Text #2

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.3496. Min Act: -0.1700

Data Index: 6675910 (Open Web Text)

Max Activating Token Index: 538

Text #3

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2905. Min Act: -0.1700

Data Index: 7686871 (Open Web Text)

Max Activating Token Index: 504

Text #4

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.3034. Min Act: -0.1700

Data Index: 8130710 (Open Web Text)

Max Activating Token Index: 890

Text #5

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2797. Min Act: -0.1700

Data Index: 4228143 (Open Web Text)

Max Activating Token Index: 1014

Text #6

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2886. Min Act: -0.1700

Data Index: 5134816 (Open Web Text)

Max Activating Token Index: 63

Text #7

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2829. Min Act: -0.1700

Data Index: 3459207 (Open Web Text)

Max Activating Token Index: 44

Text #8

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2797. Min Act: -0.1700

Data Index: 8805597 (Open Web Text)

Max Activating Token Index: 598

Text #9

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2849. Min Act: -0.1700

Data Index: 7562534 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2665. Min Act: -0.1700

Data Index: 4158706 (Open Web Text)

Max Activating Token Index: 239

Text #11

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2500. Min Act: -0.1700

Data Index: 3191211 (Open Web Text)

Max Activating Token Index: 287

Text #12

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2423. Min Act: -0.1700

Data Index: 5065159 (Open Web Text)

Max Activating Token Index: 297

Text #13

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2292. Min Act: -0.1700

Data Index: 5033997 (Open Web Text)

Max Activating Token Index: 968

Text #14

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2140. Min Act: -0.1700

Data Index: 7667256 (Open Web Text)

Max Activating Token Index: 551

Text #15

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2177. Min Act: -0.1700

Data Index: 5098482 (Open Web Text)

Max Activating Token Index: 302

Text #16

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.2329. Min Act: -0.1700

Data Index: 1020938 (Open Web Text)

Max Activating Token Index: 922

Text #17

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.1999. Min Act: -0.1700

Data Index: 1485422 (Open Web Text)

Max Activating Token Index: 739

Text #18

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.1961. Min Act: -0.1700

Data Index: 3442705 (Open Web Text)

Max Activating Token Index: 688

Text #19

Max Range: 3.4406. Min Range: -3.4406

Max Act: 3.1920. Min Act: -0.1700

Data Index: 3468918 (Open Web Text)

Max Activating Token Index: 892