Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1029 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.1702. Min Act: -0.1700

Data Index: 6741729 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.1083. Min Act: -0.1700

Data Index: 2216545 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0942. Min Act: -0.1700

Data Index: 2088366 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0713. Min Act: -0.1700

Data Index: 3281719 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0650. Min Act: -0.1700

Data Index: 7708325 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0683. Min Act: -0.1700

Data Index: 6902485 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0493. Min Act: -0.1700

Data Index: 7811662 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0479. Min Act: -0.1700

Data Index: 1119174 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0492. Min Act: -0.1700

Data Index: 4970351 (Open Web Text)

Max Activating Token Index: 5

Text #9

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0411. Min Act: -0.1700

Data Index: 4753066 (Open Web Text)

Max Activating Token Index: 5

Text #10

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0431. Min Act: -0.1700

Data Index: 6022086 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0451. Min Act: -0.1700

Data Index: 5484255 (Open Web Text)

Max Activating Token Index: 32

Text #12

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0534. Min Act: -0.1700

Data Index: 1563598 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0340. Min Act: -0.1700

Data Index: 4214112 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0361. Min Act: -0.1700

Data Index: 6024344 (Open Web Text)

Max Activating Token Index: 6

Text #15

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0239. Min Act: -0.1700

Data Index: 6545314 (Open Web Text)

Max Activating Token Index: 5

Text #16

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0267. Min Act: -0.1700

Data Index: 3657710 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0303. Min Act: -0.1700

Data Index: 1902922 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0304. Min Act: -0.1700

Data Index: 3049085 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 3.1702. Min Range: -3.1702

Max Act: 3.0332. Min Act: -0.1700

Data Index: 936645 (Open Web Text)

Max Activating Token Index: 5