Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1051 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.7124. Min Act: -0.1700

Data Index: 1927266 (Open Web Text)

Max Activating Token Index: 515

Text #1

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.6948. Min Act: -0.1700

Data Index: 7859991 (Open Web Text)

Max Activating Token Index: 73

Text #2

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.6524. Min Act: -0.1700

Data Index: 4831496 (Open Web Text)

Max Activating Token Index: 464

Text #3

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.6362. Min Act: -0.1700

Data Index: 4385235 (Open Web Text)

Max Activating Token Index: 686

Text #4

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.6543. Min Act: -0.1700

Data Index: 8658277 (Open Web Text)

Max Activating Token Index: 1016

Text #5

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.6099. Min Act: -0.1700

Data Index: 5687358 (Open Web Text)

Max Activating Token Index: 403

Text #6

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5962. Min Act: -0.1700

Data Index: 5551490 (Open Web Text)

Max Activating Token Index: 991

Text #7

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5736. Min Act: -0.1700

Data Index: 2094942 (Open Web Text)

Max Activating Token Index: 966

Text #8

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5832. Min Act: -0.1700

Data Index: 1277296 (Open Web Text)

Max Activating Token Index: 318

Text #9

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5351. Min Act: -0.1700

Data Index: 7901486 (Open Web Text)

Max Activating Token Index: 448

Text #10

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5857. Min Act: -0.1700

Data Index: 5657425 (Open Web Text)

Max Activating Token Index: 651

Text #11

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5752. Min Act: -0.1700

Data Index: 7513053 (Open Web Text)

Max Activating Token Index: 222

Text #12

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5654. Min Act: -0.1700

Data Index: 7697848 (Open Web Text)

Max Activating Token Index: 186

Text #13

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5718. Min Act: -0.1700

Data Index: 4968939 (Open Web Text)

Max Activating Token Index: 301

Text #14

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5675. Min Act: -0.1700

Data Index: 8487231 (Open Web Text)

Max Activating Token Index: 727

Text #15

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5446. Min Act: -0.1700

Data Index: 7547790 (Open Web Text)

Max Activating Token Index: 280

Text #16

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5681. Min Act: -0.1700

Data Index: 6740966 (Open Web Text)

Max Activating Token Index: 293

Text #17

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5572. Min Act: -0.1700

Data Index: 5868327 (Open Web Text)

Max Activating Token Index: 118

Text #18

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5654. Min Act: -0.1700

Data Index: 8766303 (Open Web Text)

Max Activating Token Index: 81

Text #19

Max Range: 3.7124. Min Range: -3.7124

Max Act: 3.5837. Min Act: -0.1700

Data Index: 3720401 (Open Web Text)

Max Activating Token Index: 438