Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1906 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.1743. Min Act: -0.1700

Data Index: 2245996 (Open Web Text)

Max Activating Token Index: 19

Text #1

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.1595. Min Act: -0.1700

Data Index: 8057657 (Open Web Text)

Max Activating Token Index: 15

Text #2

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.1014. Min Act: -0.1700

Data Index: 7788380 (Open Web Text)

Max Activating Token Index: 13

Text #3

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0577. Min Act: -0.1700

Data Index: 5733520 (Open Web Text)

Max Activating Token Index: 538

Text #4

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0383. Min Act: -0.1700

Data Index: 7599067 (Open Web Text)

Max Activating Token Index: 382

Text #5

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0634. Min Act: -0.1700

Data Index: 4181241 (Open Web Text)

Max Activating Token Index: 233

Text #6

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0640. Min Act: -0.1700

Data Index: 5193646 (Open Web Text)

Max Activating Token Index: 371

Text #7

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0351. Min Act: -0.1700

Data Index: 3378816 (Open Web Text)

Max Activating Token Index: 913

Text #8

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0395. Min Act: -0.1700

Data Index: 4660 (Open Web Text)

Max Activating Token Index: 178

Text #9

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0104. Min Act: -0.1700

Data Index: 1400186 (Open Web Text)

Max Activating Token Index: 593

Text #10

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0292. Min Act: -0.1700

Data Index: 6056147 (Open Web Text)

Max Activating Token Index: 772

Text #11

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0063. Min Act: -0.1700

Data Index: 3707468 (Open Web Text)

Max Activating Token Index: 971

Text #12

Max Range: 3.1743. Min Range: -3.1743

Max Act: 3.0020. Min Act: -0.1700

Data Index: 3485432 (Open Web Text)

Max Activating Token Index: 894

Text #13

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9935. Min Act: -0.1700

Data Index: 2163160 (Open Web Text)

Max Activating Token Index: 61

Text #14

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9769. Min Act: -0.1700

Data Index: 76225 (Open Web Text)

Max Activating Token Index: 157

Text #15

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9838. Min Act: -0.1700

Data Index: 6056177 (Open Web Text)

Max Activating Token Index: 773

Text #16

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9931. Min Act: -0.1700

Data Index: 1919728 (Open Web Text)

Max Activating Token Index: 167

Text #17

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9671. Min Act: -0.1700

Data Index: 1285275 (Open Web Text)

Max Activating Token Index: 57

Text #18

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9731. Min Act: -0.1700

Data Index: 2825230 (Open Web Text)

Max Activating Token Index: 605

Text #19

Max Range: 3.1743. Min Range: -3.1743

Max Act: 2.9800. Min Act: -0.1700

Data Index: 4099248 (Open Web Text)

Max Activating Token Index: 913