Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1085 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.8972. Min Act: -0.1700

Data Index: 2445188 (Open Web Text)

Max Activating Token Index: 131

Text #1

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.6655. Min Act: -0.1700

Data Index: 4824226 (Open Web Text)

Max Activating Token Index: 949

Text #2

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.6001. Min Act: -0.1700

Data Index: 4519497 (Open Web Text)

Max Activating Token Index: 922

Text #3

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.6093. Min Act: -0.1700

Data Index: 2240822 (Open Web Text)

Max Activating Token Index: 14

Text #4

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.6096. Min Act: -0.1700

Data Index: 4387679 (Open Web Text)

Max Activating Token Index: 18

Text #5

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.6299. Min Act: -0.1700

Data Index: 7086256 (Open Web Text)

Max Activating Token Index: 145

Text #6

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.5539. Min Act: -0.1700

Data Index: 5686349 (Open Web Text)

Max Activating Token Index: 63

Text #7

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.5415. Min Act: -0.1700

Data Index: 4423609 (Open Web Text)

Max Activating Token Index: 80

Text #8

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.5210. Min Act: -0.1700

Data Index: 1107407 (Open Web Text)

Max Activating Token Index: 868

Text #9

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.5064. Min Act: -0.1700

Data Index: 1662217 (Open Web Text)

Max Activating Token Index: 925

Text #10

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4651. Min Act: -0.1700

Data Index: 2735240 (Open Web Text)

Max Activating Token Index: 61

Text #11

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4799. Min Act: -0.1700

Data Index: 4423608 (Open Web Text)

Max Activating Token Index: 88

Text #12

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4576. Min Act: -0.1700

Data Index: 5296031 (Open Web Text)

Max Activating Token Index: 495

Text #13

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.5105. Min Act: -0.1700

Data Index: 1056592 (Open Web Text)

Max Activating Token Index: 935

Text #14

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4688. Min Act: -0.1700

Data Index: 8198009 (Open Web Text)

Max Activating Token Index: 664

Text #15

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4470. Min Act: -0.1700

Data Index: 5019149 (Open Web Text)

Max Activating Token Index: 76

Text #16

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4256. Min Act: -0.1700

Data Index: 8484073 (Open Web Text)

Max Activating Token Index: 331

Text #17

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4350. Min Act: -0.1700

Data Index: 5079239 (Open Web Text)

Max Activating Token Index: 82

Text #18

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4205. Min Act: -0.1700

Data Index: 3750820 (Open Web Text)

Max Activating Token Index: 556

Text #19

Max Range: 3.8972. Min Range: -3.8972

Max Act: 3.4297. Min Act: -0.1700

Data Index: 7819625 (Open Web Text)

Max Activating Token Index: 39