Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1254 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.8707. Min Act: -0.1700

Data Index: 6240366 (Open Web Text)

Max Activating Token Index: 431

Text #1

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.6930. Min Act: -0.1700

Data Index: 4618438 (Open Web Text)

Max Activating Token Index: 216

Text #2

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.6289. Min Act: -0.1700

Data Index: 304127 (Open Web Text)

Max Activating Token Index: 988

Text #3

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.5947. Min Act: -0.1700

Data Index: 610367 (Open Web Text)

Max Activating Token Index: 709

Text #4

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.5963. Min Act: -0.1700

Data Index: 6944200 (Open Web Text)

Max Activating Token Index: 687

Text #5

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.5545. Min Act: -0.1700

Data Index: 7781454 (Open Web Text)

Max Activating Token Index: 133

Text #6

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4982. Min Act: -0.1700

Data Index: 3543352 (Open Web Text)

Max Activating Token Index: 778

Text #7

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4970. Min Act: -0.1700

Data Index: 4270692 (Open Web Text)

Max Activating Token Index: 52

Text #8

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4576. Min Act: -0.1700

Data Index: 5580365 (Open Web Text)

Max Activating Token Index: 209

Text #9

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4872. Min Act: -0.1700

Data Index: 2728643 (Open Web Text)

Max Activating Token Index: 239

Text #10

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4228. Min Act: -0.1700

Data Index: 3197160 (Open Web Text)

Max Activating Token Index: 568

Text #11

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4771. Min Act: -0.1700

Data Index: 1530417 (Open Web Text)

Max Activating Token Index: 359

Text #12

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4242. Min Act: -0.1700

Data Index: 1690738 (Open Web Text)

Max Activating Token Index: 838

Text #13

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3912. Min Act: -0.1700

Data Index: 4347136 (Open Web Text)

Max Activating Token Index: 152

Text #14

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3761. Min Act: -0.1700

Data Index: 6603112 (Open Web Text)

Max Activating Token Index: 1018

Text #15

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3374. Min Act: -0.1700

Data Index: 739726 (Open Web Text)

Max Activating Token Index: 942

Text #16

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3831. Min Act: -0.1700

Data Index: 242463 (Open Web Text)

Max Activating Token Index: 744

Text #17

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3473. Min Act: -0.1700

Data Index: 4587364 (Open Web Text)

Max Activating Token Index: 861

Text #18

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.4048. Min Act: -0.1700

Data Index: 722272 (Open Web Text)

Max Activating Token Index: 938

Text #19

Max Range: 4.8707. Min Range: -4.8707

Max Act: 4.3003. Min Act: -0.1700

Data Index: 3909508 (Open Web Text)

Max Activating Token Index: 770