Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 453 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.2427. Min Act: -0.1700

Data Index: 7204851 (Open Web Text)

Max Activating Token Index: 695

Text #1

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.2411. Min Act: -0.1700

Data Index: 5969713 (Open Web Text)

Max Activating Token Index: 449

Text #2

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0405. Min Act: -0.1700

Data Index: 4783355 (Open Web Text)

Max Activating Token Index: 625

Text #3

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0567. Min Act: -0.1700

Data Index: 1974614 (Open Web Text)

Max Activating Token Index: 616

Text #4

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0895. Min Act: -0.1700

Data Index: 8770793 (Open Web Text)

Max Activating Token Index: 687

Text #5

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0219. Min Act: -0.1700

Data Index: 6985835 (Open Web Text)

Max Activating Token Index: 504

Text #6

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0452. Min Act: -0.1700

Data Index: 2647321 (Open Web Text)

Max Activating Token Index: 854

Text #7

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9657. Min Act: -0.1700

Data Index: 2746720 (Open Web Text)

Max Activating Token Index: 701

Text #8

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9739. Min Act: -0.1700

Data Index: 7550367 (Open Web Text)

Max Activating Token Index: 317

Text #9

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0133. Min Act: -0.1700

Data Index: 4872120 (Open Web Text)

Max Activating Token Index: 486

Text #10

Max Range: 5.2427. Min Range: -5.2427

Max Act: 5.0814. Min Act: -0.1700

Data Index: 7078318 (Open Web Text)

Max Activating Token Index: 575

Text #11

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9059. Min Act: -0.1700

Data Index: 847641 (Open Web Text)

Max Activating Token Index: 684

Text #12

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9594. Min Act: -0.1700

Data Index: 2356360 (Open Web Text)

Max Activating Token Index: 540

Text #13

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9432. Min Act: -0.1700

Data Index: 4758420 (Open Web Text)

Max Activating Token Index: 457

Text #14

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9747. Min Act: -0.1700

Data Index: 8012771 (Open Web Text)

Max Activating Token Index: 731

Text #15

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9646. Min Act: -0.1700

Data Index: 1976050 (Open Web Text)

Max Activating Token Index: 389

Text #16

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9017. Min Act: -0.1700

Data Index: 2930243 (Open Web Text)

Max Activating Token Index: 890

Text #17

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.9219. Min Act: -0.1700

Data Index: 4095607 (Open Web Text)

Max Activating Token Index: 734

Text #18

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.8549. Min Act: -0.1700

Data Index: 2217860 (Open Web Text)

Max Activating Token Index: 701

Text #19

Max Range: 5.2427. Min Range: -5.2427

Max Act: 4.8692. Min Act: -0.1700

Data Index: 4683361 (Open Web Text)

Max Activating Token Index: 818