Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1401 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.0403. Min Range: -2.0403

Max Act: 2.0403. Min Act: -0.1684

Data Index: 2057719 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 2.0403. Min Range: -2.0403

Max Act: 2.0026. Min Act: -0.1608

Data Index: 6007605 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9653. Min Act: -0.1700

Data Index: 4897376 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9520. Min Act: -0.1699

Data Index: 3149502 (Open Web Text)

Max Activating Token Index: 636

Text #4

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9552. Min Act: -0.1700

Data Index: 7644901 (Open Web Text)

Max Activating Token Index: 456

Text #5

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9386. Min Act: -0.1695

Data Index: 2786871 (Open Web Text)

Max Activating Token Index: 920

Text #6

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9546. Min Act: -0.1674

Data Index: 8342908 (Open Web Text)

Max Activating Token Index: 512

Text #7

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9408. Min Act: -0.1348

Data Index: 1077318 (Open Web Text)

Max Activating Token Index: 38

Text #8

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9397. Min Act: -0.1700

Data Index: 4215895 (Open Web Text)

Max Activating Token Index: 178

Text #9

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9235. Min Act: -0.1698

Data Index: 2409472 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9225. Min Act: -0.1606

Data Index: 1077320 (Open Web Text)

Max Activating Token Index: 564

Text #11

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9070. Min Act: -0.1700

Data Index: 243927 (Open Web Text)

Max Activating Token Index: 10

Text #12

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9127. Min Act: -0.1689

Data Index: 2526015 (Open Web Text)

Max Activating Token Index: 6

Text #13

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9131. Min Act: -0.1698

Data Index: 6659869 (Open Web Text)

Max Activating Token Index: 26

Text #14

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9118. Min Act: -0.1700

Data Index: 3660538 (Open Web Text)

Max Activating Token Index: 339

Text #15

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9112. Min Act: -0.1698

Data Index: 6886109 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9039. Min Act: -0.1644

Data Index: 1077317 (Open Web Text)

Max Activating Token Index: 877

Text #17

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9028. Min Act: -0.1619

Data Index: 4443046 (Open Web Text)

Max Activating Token Index: 450

Text #18

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.8981. Min Act: -0.1700

Data Index: 7282691 (Open Web Text)

Max Activating Token Index: 27

Text #19

Max Range: 2.0403. Min Range: -2.0403

Max Act: 1.9019. Min Act: -0.1322

Data Index: 1077316 (Open Web Text)

Max Activating Token Index: 8