Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1765 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.3227. Min Act: -0.1326

Data Index: 2875616 (Open Web Text)

Max Activating Token Index: 963

Text #1

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2996. Min Act: -0.1053

Data Index: 406684 (Open Web Text)

Max Activating Token Index: 966

Text #2

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2884. Min Act: -0.1071

Data Index: 40108 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2817. Min Act: -0.1141

Data Index: 1870353 (Open Web Text)

Max Activating Token Index: 455

Text #4

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2760. Min Act: -0.0731

Data Index: 276497 (Open Web Text)

Max Activating Token Index: 675

Text #5

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2719. Min Act: -0.1061

Data Index: 6146975 (Open Web Text)

Max Activating Token Index: 976

Text #6

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2765. Min Act: -0.0835

Data Index: 1356816 (Open Web Text)

Max Activating Token Index: 898

Text #7

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2687. Min Act: -0.0682

Data Index: 6611544 (Open Web Text)

Max Activating Token Index: 1006

Text #8

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2723. Min Act: -0.0763

Data Index: 8722090 (Open Web Text)

Max Activating Token Index: 411

Text #9

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2675. Min Act: -0.1177

Data Index: 8223851 (Open Web Text)

Max Activating Token Index: 524

Text #10

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2630. Min Act: -0.0723

Data Index: 5280654 (Open Web Text)

Max Activating Token Index: 957

Text #11

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2597. Min Act: -0.1294

Data Index: 8535766 (Open Web Text)

Max Activating Token Index: 550

Text #12

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2613. Min Act: -0.1166

Data Index: 2540576 (Open Web Text)

Max Activating Token Index: 1000

Text #13

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2592. Min Act: -0.1108

Data Index: 2323475 (Open Web Text)

Max Activating Token Index: 411

Text #14

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2576. Min Act: -0.0830

Data Index: 2238148 (Open Web Text)

Max Activating Token Index: 722

Text #15

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2558. Min Act: -0.1010

Data Index: 3259156 (Open Web Text)

Max Activating Token Index: 944

Text #16

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2557. Min Act: -0.0759

Data Index: 8137552 (Open Web Text)

Max Activating Token Index: 593

Text #17

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2535. Min Act: -0.0771

Data Index: 2393692 (Open Web Text)

Max Activating Token Index: 909

Text #18

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2520. Min Act: -0.0815

Data Index: 2983086 (Open Web Text)

Max Activating Token Index: 734

Text #19

Max Range: 1.3227. Min Range: -1.3227

Max Act: 1.2546. Min Act: -0.0650

Data Index: 3926494 (Open Web Text)

Max Activating Token Index: 657