Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1975 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1760. Min Range: -5.1760

Max Act: 5.1760. Min Act: -0.1700

Data Index: 7137220 (Open Web Text)

Max Activating Token Index: 550

Text #1

Max Range: 5.1760. Min Range: -5.1760

Max Act: 5.0916. Min Act: -0.1700

Data Index: 5441961 (Open Web Text)

Max Activating Token Index: 651

Text #2

Max Range: 5.1760. Min Range: -5.1760

Max Act: 5.0926. Min Act: -0.1700

Data Index: 725007 (Open Web Text)

Max Activating Token Index: 553

Text #3

Max Range: 5.1760. Min Range: -5.1760

Max Act: 5.0983. Min Act: -0.1700

Data Index: 5594782 (Open Web Text)

Max Activating Token Index: 873

Text #4

Max Range: 5.1760. Min Range: -5.1760

Max Act: 5.0518. Min Act: -0.1700

Data Index: 2859075 (Open Web Text)

Max Activating Token Index: 191

Text #5

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9546. Min Act: -0.1700

Data Index: 573528 (Open Web Text)

Max Activating Token Index: 253

Text #6

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9410. Min Act: -0.1700

Data Index: 8163914 (Open Web Text)

Max Activating Token Index: 422

Text #7

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9298. Min Act: -0.1700

Data Index: 980496 (Open Web Text)

Max Activating Token Index: 56

Text #8

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9528. Min Act: -0.1700

Data Index: 3703451 (Open Web Text)

Max Activating Token Index: 357

Text #9

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9338. Min Act: -0.1700

Data Index: 8484600 (Open Web Text)

Max Activating Token Index: 560

Text #10

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9526. Min Act: -0.1700

Data Index: 234224 (Open Web Text)

Max Activating Token Index: 995

Text #11

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9553. Min Act: -0.1700

Data Index: 4883842 (Open Web Text)

Max Activating Token Index: 737

Text #12

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9509. Min Act: -0.1700

Data Index: 2711462 (Open Web Text)

Max Activating Token Index: 642

Text #13

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9251. Min Act: -0.1700

Data Index: 6860925 (Open Web Text)

Max Activating Token Index: 426

Text #14

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9348. Min Act: -0.1700

Data Index: 3278877 (Open Web Text)

Max Activating Token Index: 301

Text #15

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9053. Min Act: -0.1700

Data Index: 5267751 (Open Web Text)

Max Activating Token Index: 615

Text #16

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.8886. Min Act: -0.1700

Data Index: 4238002 (Open Web Text)

Max Activating Token Index: 207

Text #17

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.9171. Min Act: -0.1700

Data Index: 307061 (Open Web Text)

Max Activating Token Index: 159

Text #18

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.8992. Min Act: -0.1700

Data Index: 3896790 (Open Web Text)

Max Activating Token Index: 265

Text #19

Max Range: 5.1760. Min Range: -5.1760

Max Act: 4.8905. Min Act: -0.1700

Data Index: 3402398 (Open Web Text)

Max Activating Token Index: 33