Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2147 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.2701. Min Act: -0.1700

Data Index: 7756811 (Open Web Text)

Max Activating Token Index: 5

Text #1

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.2371. Min Act: -0.1700

Data Index: 6285443 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.1227. Min Act: -0.1700

Data Index: 4278668 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.1264. Min Act: -0.1700

Data Index: 3246394 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.1408. Min Act: -0.1700

Data Index: 4362733 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.1385. Min Act: -0.1700

Data Index: 127696 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0984. Min Act: -0.1700

Data Index: 6473197 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0745. Min Act: -0.1700

Data Index: 3872431 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0553. Min Act: -0.1700

Data Index: 7753665 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0473. Min Act: -0.1700

Data Index: 8485566 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0617. Min Act: -0.1700

Data Index: 810750 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0445. Min Act: -0.1700

Data Index: 7902265 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0335. Min Act: -0.1700

Data Index: 731391 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0189. Min Act: -0.1700

Data Index: 2248433 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0268. Min Act: -0.1700

Data Index: 5711014 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0078. Min Act: -0.1700

Data Index: 4345365 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0341. Min Act: -0.1700

Data Index: 4166990 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0119. Min Act: -0.1700

Data Index: 6204602 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 5.2701. Min Range: -5.2701

Max Act: 4.9979. Min Act: -0.1700

Data Index: 2147389 (Open Web Text)

Max Activating Token Index: 9

Text #19

Max Range: 5.2701. Min Range: -5.2701

Max Act: 5.0156. Min Act: -0.1700

Data Index: 3268653 (Open Web Text)

Max Activating Token Index: 10