Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 535 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.6423. Min Act: -0.1700

Data Index: 1496155 (Open Web Text)

Max Activating Token Index: 171

Text #1

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.6130. Min Act: -0.1700

Data Index: 4714983 (Open Web Text)

Max Activating Token Index: 663

Text #2

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5338. Min Act: -0.1700

Data Index: 2184162 (Open Web Text)

Max Activating Token Index: 486

Text #3

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5582. Min Act: -0.1700

Data Index: 316444 (Open Web Text)

Max Activating Token Index: 817

Text #4

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5436. Min Act: -0.1700

Data Index: 7728676 (Open Web Text)

Max Activating Token Index: 20

Text #5

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5433. Min Act: -0.1700

Data Index: 156623 (Open Web Text)

Max Activating Token Index: 94

Text #6

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5341. Min Act: -0.1700

Data Index: 3797794 (Open Web Text)

Max Activating Token Index: 526

Text #7

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5312. Min Act: -0.1700

Data Index: 789461 (Open Web Text)

Max Activating Token Index: 117

Text #8

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4970. Min Act: -0.1700

Data Index: 4501949 (Open Web Text)

Max Activating Token Index: 800

Text #9

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5056. Min Act: -0.1700

Data Index: 383842 (Open Web Text)

Max Activating Token Index: 421

Text #10

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4872. Min Act: -0.1700

Data Index: 5677474 (Open Web Text)

Max Activating Token Index: 47

Text #11

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.5019. Min Act: -0.1700

Data Index: 451991 (Open Web Text)

Max Activating Token Index: 960

Text #12

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4504. Min Act: -0.1700

Data Index: 7774074 (Open Web Text)

Max Activating Token Index: 979

Text #13

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4605. Min Act: -0.1700

Data Index: 7273790 (Open Web Text)

Max Activating Token Index: 238

Text #14

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4004. Min Act: -0.1700

Data Index: 2973021 (Open Web Text)

Max Activating Token Index: 839

Text #15

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4065. Min Act: -0.1700

Data Index: 1058885 (Open Web Text)

Max Activating Token Index: 127

Text #16

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4169. Min Act: -0.1700

Data Index: 7070539 (Open Web Text)

Max Activating Token Index: 24

Text #17

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4352. Min Act: -0.1700

Data Index: 7853093 (Open Web Text)

Max Activating Token Index: 16

Text #18

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.3835. Min Act: -0.1700

Data Index: 5762349 (Open Web Text)

Max Activating Token Index: 709

Text #19

Max Range: 4.6423. Min Range: -4.6423

Max Act: 4.4032. Min Act: -0.1700

Data Index: 3285829 (Open Web Text)

Max Activating Token Index: 304