Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 355 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.9826. Min Act: -0.1700

Data Index: 5749447 (Open Web Text)

Max Activating Token Index: 255

Text #1

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.9632. Min Act: -0.1700

Data Index: 1676875 (Open Web Text)

Max Activating Token Index: 14

Text #2

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.9442. Min Act: -0.1698

Data Index: 1560312 (Open Web Text)

Max Activating Token Index: 1000

Text #3

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.9070. Min Act: -0.1700

Data Index: 6511007 (Open Web Text)

Max Activating Token Index: 92

Text #4

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8900. Min Act: -0.1700

Data Index: 3803291 (Open Web Text)

Max Activating Token Index: 622

Text #5

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8946. Min Act: -0.1700

Data Index: 1357051 (Open Web Text)

Max Activating Token Index: 111

Text #6

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8711. Min Act: -0.1700

Data Index: 4518240 (Open Web Text)

Max Activating Token Index: 154

Text #7

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8754. Min Act: -0.1700

Data Index: 578749 (Open Web Text)

Max Activating Token Index: 87

Text #8

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8622. Min Act: -0.1700

Data Index: 1070203 (Open Web Text)

Max Activating Token Index: 90

Text #9

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8523. Min Act: -0.1700

Data Index: 825320 (Open Web Text)

Max Activating Token Index: 483

Text #10

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8629. Min Act: -0.1700

Data Index: 4189942 (Open Web Text)

Max Activating Token Index: 203

Text #11

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8452. Min Act: -0.1700

Data Index: 6162596 (Open Web Text)

Max Activating Token Index: 617

Text #12

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8388. Min Act: -0.1700

Data Index: 873052 (Open Web Text)

Max Activating Token Index: 92

Text #13

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8442. Min Act: -0.1700

Data Index: 2448480 (Open Web Text)

Max Activating Token Index: 64

Text #14

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8535. Min Act: -0.1700

Data Index: 4395598 (Open Web Text)

Max Activating Token Index: 76

Text #15

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8478. Min Act: -0.1700

Data Index: 8516589 (Open Web Text)

Max Activating Token Index: 80

Text #16

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8386. Min Act: -0.1700

Data Index: 324292 (Open Web Text)

Max Activating Token Index: 27

Text #17

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8402. Min Act: -0.1700

Data Index: 6407362 (Open Web Text)

Max Activating Token Index: 99

Text #18

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8557. Min Act: -0.1695

Data Index: 2702945 (Open Web Text)

Max Activating Token Index: 845

Text #19

Max Range: 1.9826. Min Range: -1.9826

Max Act: 1.8420. Min Act: -0.1700

Data Index: 4214343 (Open Web Text)

Max Activating Token Index: 82