Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2140 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.9546. Min Act: -0.1700

Data Index: 2703444 (Open Web Text)

Max Activating Token Index: 235

Text #1

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8951. Min Act: -0.1700

Data Index: 998718 (Open Web Text)

Max Activating Token Index: 929

Text #2

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8373. Min Act: -0.1700

Data Index: 3311613 (Open Web Text)

Max Activating Token Index: 20

Text #3

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8473. Min Act: -0.1700

Data Index: 1333182 (Open Web Text)

Max Activating Token Index: 622

Text #4

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8290. Min Act: -0.1700

Data Index: 7112702 (Open Web Text)

Max Activating Token Index: 40

Text #5

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8287. Min Act: -0.1700

Data Index: 7957097 (Open Web Text)

Max Activating Token Index: 165

Text #6

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.8263. Min Act: -0.1700

Data Index: 1333180 (Open Web Text)

Max Activating Token Index: 864

Text #7

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7976. Min Act: -0.1700

Data Index: 2523030 (Open Web Text)

Max Activating Token Index: 107

Text #8

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7798. Min Act: -0.1700

Data Index: 8510360 (Open Web Text)

Max Activating Token Index: 198

Text #9

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7812. Min Act: -0.1700

Data Index: 4356107 (Open Web Text)

Max Activating Token Index: 828

Text #10

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7842. Min Act: -0.1700

Data Index: 2749650 (Open Web Text)

Max Activating Token Index: 757

Text #11

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7433. Min Act: -0.1700

Data Index: 3496132 (Open Web Text)

Max Activating Token Index: 1015

Text #12

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7681. Min Act: -0.1700

Data Index: 5370353 (Open Web Text)

Max Activating Token Index: 36

Text #13

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7745. Min Act: -0.1700

Data Index: 3130330 (Open Web Text)

Max Activating Token Index: 519

Text #14

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7603. Min Act: -0.1700

Data Index: 7170812 (Open Web Text)

Max Activating Token Index: 982

Text #15

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7585. Min Act: -0.1700

Data Index: 495118 (Open Web Text)

Max Activating Token Index: 147

Text #16

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7487. Min Act: -0.1700

Data Index: 135550 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7592. Min Act: -0.1700

Data Index: 1327160 (Open Web Text)

Max Activating Token Index: 274

Text #18

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7963. Min Act: -0.1700

Data Index: 2635833 (Open Web Text)

Max Activating Token Index: 989

Text #19

Max Range: 3.9546. Min Range: -3.9546

Max Act: 3.7681. Min Act: -0.1700

Data Index: 2269563 (Open Web Text)

Max Activating Token Index: 629