Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2909 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.5403. Min Act: -0.1700

Data Index: 82170 (Open Web Text)

Max Activating Token Index: 428

Text #1

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.5099. Min Act: -0.1700

Data Index: 6222859 (Open Web Text)

Max Activating Token Index: 562

Text #2

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4948. Min Act: -0.1700

Data Index: 6510258 (Open Web Text)

Max Activating Token Index: 444

Text #3

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4526. Min Act: -0.1700

Data Index: 7532274 (Open Web Text)

Max Activating Token Index: 725

Text #4

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4647. Min Act: -0.1700

Data Index: 7213880 (Open Web Text)

Max Activating Token Index: 634

Text #5

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4569. Min Act: -0.1700

Data Index: 4937798 (Open Web Text)

Max Activating Token Index: 428

Text #6

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4527. Min Act: -0.1700

Data Index: 311353 (Open Web Text)

Max Activating Token Index: 358

Text #7

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4386. Min Act: -0.1700

Data Index: 1449268 (Open Web Text)

Max Activating Token Index: 107

Text #8

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4468. Min Act: -0.1700

Data Index: 1835467 (Open Web Text)

Max Activating Token Index: 361

Text #9

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4371. Min Act: -0.1700

Data Index: 768786 (Open Web Text)

Max Activating Token Index: 932

Text #10

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4203. Min Act: -0.1700

Data Index: 8139169 (Open Web Text)

Max Activating Token Index: 607

Text #11

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4361. Min Act: -0.1700

Data Index: 2614275 (Open Web Text)

Max Activating Token Index: 968

Text #12

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4142. Min Act: -0.1700

Data Index: 5707217 (Open Web Text)

Max Activating Token Index: 440

Text #13

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.3998. Min Act: -0.1700

Data Index: 973194 (Open Web Text)

Max Activating Token Index: 495

Text #14

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4127. Min Act: -0.1700

Data Index: 1265415 (Open Web Text)

Max Activating Token Index: 947

Text #15

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4172. Min Act: -0.1700

Data Index: 6972116 (Open Web Text)

Max Activating Token Index: 530

Text #16

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4096. Min Act: -0.1700

Data Index: 6520575 (Open Web Text)

Max Activating Token Index: 386

Text #17

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.3884. Min Act: -0.1700

Data Index: 2565091 (Open Web Text)

Max Activating Token Index: 532

Text #18

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4036. Min Act: -0.1700

Data Index: 5404651 (Open Web Text)

Max Activating Token Index: 404

Text #19

Max Range: 2.5403. Min Range: -2.5403

Max Act: 2.4075. Min Act: -0.1700

Data Index: 100437 (Open Web Text)

Max Activating Token Index: 469