Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1883 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.5498. Min Act: -0.1700

Data Index: 713159 (Open Web Text)

Max Activating Token Index: 443

Text #1

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.4462. Min Act: -0.1700

Data Index: 6685991 (Open Web Text)

Max Activating Token Index: 691

Text #2

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.4215. Min Act: -0.1700

Data Index: 8353931 (Open Web Text)

Max Activating Token Index: 37

Text #3

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3875. Min Act: -0.1700

Data Index: 7722576 (Open Web Text)

Max Activating Token Index: 26

Text #4

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3974. Min Act: -0.1700

Data Index: 4906054 (Open Web Text)

Max Activating Token Index: 458

Text #5

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3781. Min Act: -0.1700

Data Index: 2664464 (Open Web Text)

Max Activating Token Index: 730

Text #6

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3518. Min Act: -0.1700

Data Index: 3786829 (Open Web Text)

Max Activating Token Index: 58

Text #7

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3583. Min Act: -0.1700

Data Index: 7415772 (Open Web Text)

Max Activating Token Index: 488

Text #8

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3441. Min Act: -0.1700

Data Index: 8319867 (Open Web Text)

Max Activating Token Index: 137

Text #9

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3393. Min Act: -0.1700

Data Index: 4336690 (Open Web Text)

Max Activating Token Index: 378

Text #10

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3415. Min Act: -0.1700

Data Index: 1770587 (Open Web Text)

Max Activating Token Index: 1011

Text #11

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3233. Min Act: -0.1700

Data Index: 6838607 (Open Web Text)

Max Activating Token Index: 400

Text #12

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3029. Min Act: -0.1700

Data Index: 178962 (Open Web Text)

Max Activating Token Index: 515

Text #13

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3281. Min Act: -0.1700

Data Index: 5380459 (Open Web Text)

Max Activating Token Index: 5

Text #14

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3134. Min Act: -0.1700

Data Index: 147150 (Open Web Text)

Max Activating Token Index: 16

Text #15

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3052. Min Act: -0.1700

Data Index: 1353329 (Open Web Text)

Max Activating Token Index: 160

Text #16

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3203. Min Act: -0.1700

Data Index: 3535301 (Open Web Text)

Max Activating Token Index: 31

Text #17

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3171. Min Act: -0.1700

Data Index: 422805 (Open Web Text)

Max Activating Token Index: 256

Text #18

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3287. Min Act: -0.1700

Data Index: 7787289 (Open Web Text)

Max Activating Token Index: 429

Text #19

Max Range: 3.5498. Min Range: -3.5498

Max Act: 3.3279. Min Act: -0.1700

Data Index: 3609393 (Open Web Text)

Max Activating Token Index: 63