Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2058 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4838. Min Act: -0.1700

Data Index: 2591022 (Open Web Text)

Max Activating Token Index: 546

Text #1

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.5273. Min Act: -0.1700

Data Index: 5165167 (Open Web Text)

Max Activating Token Index: 437

Text #2

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4634. Min Act: -0.1700

Data Index: 2195864 (Open Web Text)

Max Activating Token Index: 616

Text #3

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4692. Min Act: -0.1700

Data Index: 2195865 (Open Web Text)

Max Activating Token Index: 156

Text #4

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4240. Min Act: -0.1700

Data Index: 4831420 (Open Web Text)

Max Activating Token Index: 761

Text #5

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3959. Min Act: -0.1700

Data Index: 5714677 (Open Web Text)

Max Activating Token Index: 871

Text #6

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4031. Min Act: -0.1700

Data Index: 540318 (Open Web Text)

Max Activating Token Index: 706

Text #7

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.4182. Min Act: -0.1700

Data Index: 5072710 (Open Web Text)

Max Activating Token Index: 1014

Text #8

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3496. Min Act: -0.1700

Data Index: 6356658 (Open Web Text)

Max Activating Token Index: 413

Text #9

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3561. Min Act: -0.1700

Data Index: 8334406 (Open Web Text)

Max Activating Token Index: 142

Text #10

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3542. Min Act: -0.1700

Data Index: 4356306 (Open Web Text)

Max Activating Token Index: 242

Text #11

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3543. Min Act: -0.1700

Data Index: 231092 (Open Web Text)

Max Activating Token Index: 865

Text #12

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3114. Min Act: -0.1700

Data Index: 3170894 (Open Web Text)

Max Activating Token Index: 866

Text #13

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3248. Min Act: -0.1700

Data Index: 3606245 (Open Web Text)

Max Activating Token Index: 337

Text #14

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3328. Min Act: -0.1700

Data Index: 8280620 (Open Web Text)

Max Activating Token Index: 596

Text #15

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3502. Min Act: -0.1700

Data Index: 5258600 (Open Web Text)

Max Activating Token Index: 607

Text #16

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.3136. Min Act: -0.1700

Data Index: 2682640 (Open Web Text)

Max Activating Token Index: 680

Text #17

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.2437. Min Act: -0.1700

Data Index: 6984309 (Open Web Text)

Max Activating Token Index: 686

Text #18

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.2875. Min Act: -0.1700

Data Index: 7936952 (Open Web Text)

Max Activating Token Index: 704

Text #19

Max Range: 4.5273. Min Range: -4.5273

Max Act: 4.2879. Min Act: -0.1700

Data Index: 2682641 (Open Web Text)

Max Activating Token Index: 78