Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 506 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0733. Min Range: -5.0733

Max Act: 5.0733. Min Act: -0.1700

Data Index: 4974423 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 5.0733. Min Range: -5.0733

Max Act: 5.0556. Min Act: -0.1700

Data Index: 2324632 (Open Web Text)

Max Activating Token Index: 10

Text #2

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.8612. Min Act: -0.1700

Data Index: 7348208 (Open Web Text)

Max Activating Token Index: 176

Text #3

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.7207. Min Act: -0.1700

Data Index: 3601522 (Open Web Text)

Max Activating Token Index: 4

Text #4

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.7040. Min Act: -0.1700

Data Index: 4485268 (Open Web Text)

Max Activating Token Index: 651

Text #5

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.7236. Min Act: -0.1700

Data Index: 2714803 (Open Web Text)

Max Activating Token Index: 8

Text #6

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6994. Min Act: -0.1700

Data Index: 252163 (Open Web Text)

Max Activating Token Index: 25

Text #7

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6624. Min Act: -0.1700

Data Index: 7348209 (Open Web Text)

Max Activating Token Index: 119

Text #8

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6325. Min Act: -0.1700

Data Index: 6972326 (Open Web Text)

Max Activating Token Index: 843

Text #9

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6564. Min Act: -0.1700

Data Index: 5278830 (Open Web Text)

Max Activating Token Index: 35

Text #10

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6445. Min Act: -0.1700

Data Index: 3601528 (Open Web Text)

Max Activating Token Index: 37

Text #11

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6113. Min Act: -0.1700

Data Index: 2879877 (Open Web Text)

Max Activating Token Index: 81

Text #12

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5660. Min Act: -0.1700

Data Index: 6702213 (Open Web Text)

Max Activating Token Index: 10

Text #13

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5888. Min Act: -0.1700

Data Index: 5446746 (Open Web Text)

Max Activating Token Index: 28

Text #14

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.6177. Min Act: -0.1700

Data Index: 254741 (Open Web Text)

Max Activating Token Index: 815

Text #15

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5756. Min Act: -0.1700

Data Index: 694544 (Open Web Text)

Max Activating Token Index: 11

Text #16

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5432. Min Act: -0.1700

Data Index: 8367265 (Open Web Text)

Max Activating Token Index: 300

Text #17

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5568. Min Act: -0.1700

Data Index: 6947179 (Open Web Text)

Max Activating Token Index: 8

Text #18

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5301. Min Act: -0.1700

Data Index: 6913905 (Open Web Text)

Max Activating Token Index: 126

Text #19

Max Range: 5.0733. Min Range: -5.0733

Max Act: 4.5374. Min Act: -0.1700

Data Index: 4396479 (Open Web Text)

Max Activating Token Index: 14