Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1055 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.5840. Min Act: -0.1700

Data Index: 3295151 (Open Web Text)

Max Activating Token Index: 583

Text #1

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.5497. Min Act: -0.1700

Data Index: 4148282 (Open Web Text)

Max Activating Token Index: 950

Text #2

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.5214. Min Act: -0.1700

Data Index: 5993880 (Open Web Text)

Max Activating Token Index: 422

Text #3

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.4889. Min Act: -0.1700

Data Index: 6550566 (Open Web Text)

Max Activating Token Index: 707

Text #4

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.4508. Min Act: -0.1700

Data Index: 4532371 (Open Web Text)

Max Activating Token Index: 265

Text #5

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3993. Min Act: -0.1700

Data Index: 2142832 (Open Web Text)

Max Activating Token Index: 855

Text #6

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3726. Min Act: -0.1700

Data Index: 5260609 (Open Web Text)

Max Activating Token Index: 27

Text #7

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3365. Min Act: -0.1700

Data Index: 516694 (Open Web Text)

Max Activating Token Index: 173

Text #8

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3447. Min Act: -0.1700

Data Index: 53608 (Open Web Text)

Max Activating Token Index: 93

Text #9

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3668. Min Act: -0.1700

Data Index: 7557983 (Open Web Text)

Max Activating Token Index: 495

Text #10

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3288. Min Act: -0.1700

Data Index: 3878114 (Open Web Text)

Max Activating Token Index: 128

Text #11

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2955. Min Act: -0.1700

Data Index: 3573243 (Open Web Text)

Max Activating Token Index: 763

Text #12

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.3140. Min Act: -0.1700

Data Index: 5073331 (Open Web Text)

Max Activating Token Index: 332

Text #13

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2842. Min Act: -0.1700

Data Index: 2264692 (Open Web Text)

Max Activating Token Index: 434

Text #14

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2966. Min Act: -0.1700

Data Index: 4019242 (Open Web Text)

Max Activating Token Index: 973

Text #15

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2423. Min Act: -0.1700

Data Index: 1895711 (Open Web Text)

Max Activating Token Index: 456

Text #16

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2418. Min Act: -0.1700

Data Index: 3637828 (Open Web Text)

Max Activating Token Index: 446

Text #17

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2583. Min Act: -0.1700

Data Index: 1682650 (Open Web Text)

Max Activating Token Index: 644

Text #18

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2563. Min Act: -0.1700

Data Index: 6496336 (Open Web Text)

Max Activating Token Index: 578

Text #19

Max Range: 4.5840. Min Range: -4.5840

Max Act: 4.2142. Min Act: -0.1700

Data Index: 3986470 (Open Web Text)

Max Activating Token Index: 940