Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 601 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.2271. Min Act: -0.1700

Data Index: 5097281 (Open Web Text)

Max Activating Token Index: 247

Text #1

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.0799. Min Act: -0.1700

Data Index: 2311050 (Open Web Text)

Max Activating Token Index: 1013

Text #2

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.0908. Min Act: -0.1700

Data Index: 5331699 (Open Web Text)

Max Activating Token Index: 659

Text #3

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.0060. Min Act: -0.1700

Data Index: 4537608 (Open Web Text)

Max Activating Token Index: 376

Text #4

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.0151. Min Act: -0.1700

Data Index: 538292 (Open Web Text)

Max Activating Token Index: 143

Text #5

Max Range: 4.2271. Min Range: -4.2271

Max Act: 4.0186. Min Act: -0.1700

Data Index: 5710511 (Open Web Text)

Max Activating Token Index: 595

Text #6

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9930. Min Act: -0.1700

Data Index: 2986811 (Open Web Text)

Max Activating Token Index: 348

Text #7

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9383. Min Act: -0.1700

Data Index: 1945379 (Open Web Text)

Max Activating Token Index: 868

Text #8

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9258. Min Act: -0.1700

Data Index: 6299695 (Open Web Text)

Max Activating Token Index: 779

Text #9

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9876. Min Act: -0.1700

Data Index: 1655293 (Open Web Text)

Max Activating Token Index: 261

Text #10

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9628. Min Act: -0.1700

Data Index: 4254120 (Open Web Text)

Max Activating Token Index: 100

Text #11

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9742. Min Act: -0.1700

Data Index: 3750724 (Open Web Text)

Max Activating Token Index: 75

Text #12

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9707. Min Act: -0.1700

Data Index: 603825 (Open Web Text)

Max Activating Token Index: 31

Text #13

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9635. Min Act: -0.1700

Data Index: 231215 (Open Web Text)

Max Activating Token Index: 69

Text #14

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9535. Min Act: -0.1700

Data Index: 2153095 (Open Web Text)

Max Activating Token Index: 742

Text #15

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9710. Min Act: -0.1700

Data Index: 3220336 (Open Web Text)

Max Activating Token Index: 934

Text #16

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9543. Min Act: -0.1700

Data Index: 4542360 (Open Web Text)

Max Activating Token Index: 181

Text #17

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9927. Min Act: -0.1700

Data Index: 2793201 (Open Web Text)

Max Activating Token Index: 480

Text #18

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9949. Min Act: -0.1700

Data Index: 4357733 (Open Web Text)

Max Activating Token Index: 726

Text #19

Max Range: 4.2271. Min Range: -4.2271

Max Act: 3.9308. Min Act: -0.1700

Data Index: 138009 (Open Web Text)

Max Activating Token Index: 622