Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2024 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.1077. Min Act: -0.1700

Data Index: 7022363 (Open Web Text)

Max Activating Token Index: 8

Text #1

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0236. Min Act: -0.1700

Data Index: 2602103 (Open Web Text)

Max Activating Token Index: 469

Text #2

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0087. Min Act: -0.1700

Data Index: 2540090 (Open Web Text)

Max Activating Token Index: 649

Text #3

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0577. Min Act: -0.1700

Data Index: 4982232 (Open Web Text)

Max Activating Token Index: 378

Text #4

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0244. Min Act: -0.1700

Data Index: 7457023 (Open Web Text)

Max Activating Token Index: 669

Text #5

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0307. Min Act: -0.1700

Data Index: 6716211 (Open Web Text)

Max Activating Token Index: 647

Text #6

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0337. Min Act: -0.1700

Data Index: 3156587 (Open Web Text)

Max Activating Token Index: 316

Text #7

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9996. Min Act: -0.1700

Data Index: 7609328 (Open Web Text)

Max Activating Token Index: 544

Text #8

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9981. Min Act: -0.1700

Data Index: 2151849 (Open Web Text)

Max Activating Token Index: 277

Text #9

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0016. Min Act: -0.1700

Data Index: 585034 (Open Web Text)

Max Activating Token Index: 248

Text #10

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0039. Min Act: -0.1700

Data Index: 6150949 (Open Web Text)

Max Activating Token Index: 516

Text #11

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0178. Min Act: -0.1700

Data Index: 1625950 (Open Web Text)

Max Activating Token Index: 1009

Text #12

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9920. Min Act: -0.1700

Data Index: 3418087 (Open Web Text)

Max Activating Token Index: 340

Text #13

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0026. Min Act: -0.1700

Data Index: 3181611 (Open Web Text)

Max Activating Token Index: 400

Text #14

Max Range: 5.1077. Min Range: -5.1077

Max Act: 5.0011. Min Act: -0.1700

Data Index: 3150353 (Open Web Text)

Max Activating Token Index: 331

Text #15

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9588. Min Act: -0.1700

Data Index: 677398 (Open Web Text)

Max Activating Token Index: 985

Text #16

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9815. Min Act: -0.1700

Data Index: 3689547 (Open Web Text)

Max Activating Token Index: 527

Text #17

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9887. Min Act: -0.1700

Data Index: 3884728 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9768. Min Act: -0.1700

Data Index: 3967470 (Open Web Text)

Max Activating Token Index: 21

Text #19

Max Range: 5.1077. Min Range: -5.1077

Max Act: 4.9812. Min Act: -0.1700

Data Index: 4146307 (Open Web Text)

Max Activating Token Index: 560