Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2145 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.5232. Min Act: -0.1700

Data Index: 6694996 (Open Web Text)

Max Activating Token Index: 629

Text #1

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.4556. Min Act: -0.1700

Data Index: 6649480 (Open Web Text)

Max Activating Token Index: 610

Text #2

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.4661. Min Act: -0.1700

Data Index: 6990151 (Open Web Text)

Max Activating Token Index: 870

Text #3

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3970. Min Act: -0.1700

Data Index: 6138983 (Open Web Text)

Max Activating Token Index: 479

Text #4

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3997. Min Act: -0.1700

Data Index: 5598199 (Open Web Text)

Max Activating Token Index: 663

Text #5

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3756. Min Act: -0.1700

Data Index: 4814270 (Open Web Text)

Max Activating Token Index: 788

Text #6

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3940. Min Act: -0.1700

Data Index: 3443101 (Open Web Text)

Max Activating Token Index: 511

Text #7

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.4007. Min Act: -0.1700

Data Index: 211041 (Open Web Text)

Max Activating Token Index: 22

Text #8

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3653. Min Act: -0.1700

Data Index: 3757570 (Open Web Text)

Max Activating Token Index: 626

Text #9

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3687. Min Act: -0.1700

Data Index: 7643224 (Open Web Text)

Max Activating Token Index: 185

Text #10

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3451. Min Act: -0.1700

Data Index: 6484947 (Open Web Text)

Max Activating Token Index: 832

Text #11

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3971. Min Act: -0.1700

Data Index: 1186297 (Open Web Text)

Max Activating Token Index: 626

Text #12

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3480. Min Act: -0.1700

Data Index: 465854 (Open Web Text)

Max Activating Token Index: 399

Text #13

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3499. Min Act: -0.1700

Data Index: 3827079 (Open Web Text)

Max Activating Token Index: 469

Text #14

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3331. Min Act: -0.1700

Data Index: 3575726 (Open Web Text)

Max Activating Token Index: 789

Text #15

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3623. Min Act: -0.1700

Data Index: 4886610 (Open Web Text)

Max Activating Token Index: 81

Text #16

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3147. Min Act: -0.1700

Data Index: 4757840 (Open Web Text)

Max Activating Token Index: 469

Text #17

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.2844. Min Act: -0.1700

Data Index: 2557666 (Open Web Text)

Max Activating Token Index: 842

Text #18

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.2797. Min Act: -0.1700

Data Index: 2962412 (Open Web Text)

Max Activating Token Index: 814

Text #19

Max Range: 4.5232. Min Range: -4.5232

Max Act: 4.3170. Min Act: -0.1700

Data Index: 3715304 (Open Web Text)

Max Activating Token Index: 277