Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2578 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.8321. Min Act: -0.1700

Data Index: 4542872 (Open Web Text)

Max Activating Token Index: 27

Text #1

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.8154. Min Act: -0.1700

Data Index: 8765149 (Open Web Text)

Max Activating Token Index: 67

Text #2

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.8137. Min Act: -0.1700

Data Index: 7680177 (Open Web Text)

Max Activating Token Index: 653

Text #3

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7864. Min Act: -0.1700

Data Index: 87084 (Open Web Text)

Max Activating Token Index: 904

Text #4

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7939. Min Act: -0.1700

Data Index: 8465289 (Open Web Text)

Max Activating Token Index: 6

Text #5

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.8067. Min Act: -0.1700

Data Index: 529255 (Open Web Text)

Max Activating Token Index: 7

Text #6

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.8031. Min Act: -0.1700

Data Index: 4866959 (Open Web Text)

Max Activating Token Index: 897

Text #7

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7912. Min Act: -0.1700

Data Index: 5830843 (Open Web Text)

Max Activating Token Index: 726

Text #8

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7739. Min Act: -0.1700

Data Index: 3230763 (Open Web Text)

Max Activating Token Index: 9

Text #9

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7545. Min Act: -0.1700

Data Index: 730415 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7479. Min Act: -0.1700

Data Index: 3089226 (Open Web Text)

Max Activating Token Index: 1019

Text #11

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7452. Min Act: -0.1700

Data Index: 8348637 (Open Web Text)

Max Activating Token Index: 467

Text #12

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7459. Min Act: -0.1700

Data Index: 6720496 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7383. Min Act: -0.1700

Data Index: 8534118 (Open Web Text)

Max Activating Token Index: 13

Text #14

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7393. Min Act: -0.1700

Data Index: 2292359 (Open Web Text)

Max Activating Token Index: 409

Text #15

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7262. Min Act: -0.1700

Data Index: 2976245 (Open Web Text)

Max Activating Token Index: 937

Text #16

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7261. Min Act: -0.1700

Data Index: 6541690 (Open Web Text)

Max Activating Token Index: 20

Text #17

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7253. Min Act: -0.1700

Data Index: 4367309 (Open Web Text)

Max Activating Token Index: 361

Text #18

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7295. Min Act: -0.1700

Data Index: 504113 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 1.8321. Min Range: -1.8321

Max Act: 1.7321. Min Act: -0.1700

Data Index: 8813923 (Open Web Text)

Max Activating Token Index: 217