Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 154 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.5535. Min Act: -0.1700

Data Index: 2566623 (Open Web Text)

Max Activating Token Index: 354

Text #1

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.2464. Min Act: -0.1700

Data Index: 2704097 (Open Web Text)

Max Activating Token Index: 536

Text #2

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.2001. Min Act: -0.1700

Data Index: 3167107 (Open Web Text)

Max Activating Token Index: 468

Text #3

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1902. Min Act: -0.1700

Data Index: 5036478 (Open Web Text)

Max Activating Token Index: 6

Text #4

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1842. Min Act: -0.1700

Data Index: 2349564 (Open Web Text)

Max Activating Token Index: 30

Text #5

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1568. Min Act: -0.1700

Data Index: 2971373 (Open Web Text)

Max Activating Token Index: 780

Text #6

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1496. Min Act: -0.1700

Data Index: 1709445 (Open Web Text)

Max Activating Token Index: 264

Text #7

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1268. Min Act: -0.1700

Data Index: 5895473 (Open Web Text)

Max Activating Token Index: 634

Text #8

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1307. Min Act: -0.1700

Data Index: 3945247 (Open Web Text)

Max Activating Token Index: 325

Text #9

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1179. Min Act: -0.1700

Data Index: 2269352 (Open Web Text)

Max Activating Token Index: 25

Text #10

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1318. Min Act: -0.1700

Data Index: 8683301 (Open Web Text)

Max Activating Token Index: 881

Text #11

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.1062. Min Act: -0.1700

Data Index: 6083324 (Open Web Text)

Max Activating Token Index: 16

Text #12

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0924. Min Act: -0.1700

Data Index: 4866287 (Open Web Text)

Max Activating Token Index: 25

Text #13

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0929. Min Act: -0.1700

Data Index: 76390 (Open Web Text)

Max Activating Token Index: 246

Text #14

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0923. Min Act: -0.1700

Data Index: 1921145 (Open Web Text)

Max Activating Token Index: 432

Text #15

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0505. Min Act: -0.1700

Data Index: 5702929 (Open Web Text)

Max Activating Token Index: 735

Text #16

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0690. Min Act: -0.1700

Data Index: 6372151 (Open Web Text)

Max Activating Token Index: 195

Text #17

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0518. Min Act: -0.1700

Data Index: 7622209 (Open Web Text)

Max Activating Token Index: 845

Text #18

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0477. Min Act: -0.1700

Data Index: 3563681 (Open Web Text)

Max Activating Token Index: 918

Text #19

Max Range: 3.5535. Min Range: -3.5535

Max Act: 3.0517. Min Act: -0.1700

Data Index: 6977795 (Open Web Text)

Max Activating Token Index: 635