Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1531 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.1180. Min Range: -6.1180

Max Act: 6.0496. Min Act: -0.1700

Data Index: 7675979 (Open Web Text)

Max Activating Token Index: 240

Text #1

Max Range: 6.1180. Min Range: -6.1180

Max Act: 6.0563. Min Act: -0.1700

Data Index: 222881 (Open Web Text)

Max Activating Token Index: 359

Text #2

Max Range: 6.1180. Min Range: -6.1180

Max Act: 6.1180. Min Act: -0.1700

Data Index: 3053454 (Open Web Text)

Max Activating Token Index: 764

Text #3

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.9675. Min Act: -0.1700

Data Index: 1384340 (Open Web Text)

Max Activating Token Index: 83

Text #4

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.9599. Min Act: -0.1700

Data Index: 2082031 (Open Web Text)

Max Activating Token Index: 129

Text #5

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.9000. Min Act: -0.1700

Data Index: 5653067 (Open Web Text)

Max Activating Token Index: 699

Text #6

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7960. Min Act: -0.1700

Data Index: 7632545 (Open Web Text)

Max Activating Token Index: 386

Text #7

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7807. Min Act: -0.1700

Data Index: 4850321 (Open Web Text)

Max Activating Token Index: 530

Text #8

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.8137. Min Act: -0.1700

Data Index: 7314534 (Open Web Text)

Max Activating Token Index: 893

Text #9

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.8131. Min Act: -0.1700

Data Index: 4269238 (Open Web Text)

Max Activating Token Index: 173

Text #10

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.8740. Min Act: -0.1700

Data Index: 1143266 (Open Web Text)

Max Activating Token Index: 796

Text #11

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7693. Min Act: -0.1700

Data Index: 6152467 (Open Web Text)

Max Activating Token Index: 79

Text #12

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.8260. Min Act: -0.1700

Data Index: 7539740 (Open Web Text)

Max Activating Token Index: 598

Text #13

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.8326. Min Act: -0.1700

Data Index: 8488960 (Open Web Text)

Max Activating Token Index: 452

Text #14

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7802. Min Act: -0.1700

Data Index: 3021988 (Open Web Text)

Max Activating Token Index: 119

Text #15

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7664. Min Act: -0.1700

Data Index: 7021208 (Open Web Text)

Max Activating Token Index: 560

Text #16

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7281. Min Act: -0.1700

Data Index: 7104458 (Open Web Text)

Max Activating Token Index: 285

Text #17

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7421. Min Act: -0.1700

Data Index: 4680404 (Open Web Text)

Max Activating Token Index: 592

Text #18

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7411. Min Act: -0.1700

Data Index: 1547752 (Open Web Text)

Max Activating Token Index: 798

Text #19

Max Range: 6.1180. Min Range: -6.1180

Max Act: 5.7332. Min Act: -0.1700

Data Index: 6865003 (Open Web Text)

Max Activating Token Index: 38