Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 44 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.3182. Min Act: -0.0810

Data Index: 419622 (Open Web Text)

Max Activating Token Index: 502

Text #1

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.3004. Min Act: -0.0330

Data Index: 2679110 (Open Web Text)

Max Activating Token Index: 1012

Text #2

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.3005. Min Act: 0.0069

Data Index: 6324537 (Open Web Text)

Max Activating Token Index: 992

Text #3

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2975. Min Act: -0.0289

Data Index: 3496106 (Open Web Text)

Max Activating Token Index: 1010

Text #4

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2954. Min Act: -0.0831

Data Index: 7038712 (Open Web Text)

Max Activating Token Index: 998

Text #5

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2824. Min Act: -0.0767

Data Index: 4175674 (Open Web Text)

Max Activating Token Index: 9

Text #6

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2729. Min Act: -0.0883

Data Index: 256057 (Open Web Text)

Max Activating Token Index: 993

Text #7

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2692. Min Act: -0.0788

Data Index: 3158685 (Open Web Text)

Max Activating Token Index: 973

Text #8

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2691. Min Act: -0.0625

Data Index: 1502771 (Open Web Text)

Max Activating Token Index: 1015

Text #9

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2722. Min Act: -0.0931

Data Index: 8258369 (Open Web Text)

Max Activating Token Index: 605

Text #10

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2661. Min Act: -0.0848

Data Index: 1394797 (Open Web Text)

Max Activating Token Index: 1006

Text #11

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2495. Min Act: -0.0791

Data Index: 2321232 (Open Web Text)

Max Activating Token Index: 983

Text #12

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2531. Min Act: -0.0602

Data Index: 5305180 (Open Web Text)

Max Activating Token Index: 1007

Text #13

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2566. Min Act: 0.0043

Data Index: 3011731 (Open Web Text)

Max Activating Token Index: 984

Text #14

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2537. Min Act: -0.0102

Data Index: 7629062 (Open Web Text)

Max Activating Token Index: 1020

Text #15

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2503. Min Act: -0.1528

Data Index: 7443393 (Open Web Text)

Max Activating Token Index: 1020

Text #16

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2645. Min Act: -0.0685

Data Index: 7376414 (Open Web Text)

Max Activating Token Index: 1005

Text #17

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2606. Min Act: -0.0599

Data Index: 3521159 (Open Web Text)

Max Activating Token Index: 415

Text #18

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2585. Min Act: -0.0669

Data Index: 5779419 (Open Web Text)

Max Activating Token Index: 650

Text #19

Max Range: 1.3182. Min Range: -1.3182

Max Act: 1.2553. Min Act: -0.0210

Data Index: 122344 (Open Web Text)

Max Activating Token Index: 619