Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 368 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.3968. Min Act: -0.1700

Data Index: 6397408 (Open Web Text)

Max Activating Token Index: 167

Text #1

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.3495. Min Act: -0.1700

Data Index: 6519285 (Open Web Text)

Max Activating Token Index: 136

Text #2

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.3418. Min Act: -0.1700

Data Index: 7792474 (Open Web Text)

Max Activating Token Index: 594

Text #3

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2691. Min Act: -0.1700

Data Index: 5255710 (Open Web Text)

Max Activating Token Index: 795

Text #4

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2944. Min Act: -0.1700

Data Index: 6785551 (Open Web Text)

Max Activating Token Index: 614

Text #5

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2807. Min Act: -0.1700

Data Index: 5451088 (Open Web Text)

Max Activating Token Index: 130

Text #6

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2946. Min Act: -0.1700

Data Index: 4805535 (Open Web Text)

Max Activating Token Index: 726

Text #7

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2561. Min Act: -0.1700

Data Index: 6716814 (Open Web Text)

Max Activating Token Index: 166

Text #8

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2428. Min Act: -0.1700

Data Index: 8430171 (Open Web Text)

Max Activating Token Index: 362

Text #9

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2407. Min Act: -0.1700

Data Index: 3740876 (Open Web Text)

Max Activating Token Index: 130

Text #10

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2197. Min Act: -0.1700

Data Index: 4888082 (Open Web Text)

Max Activating Token Index: 401

Text #11

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2205. Min Act: -0.1700

Data Index: 2214869 (Open Web Text)

Max Activating Token Index: 397

Text #12

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2161. Min Act: -0.1700

Data Index: 4745000 (Open Web Text)

Max Activating Token Index: 897

Text #13

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1583. Min Act: -0.1700

Data Index: 4208711 (Open Web Text)

Max Activating Token Index: 408

Text #14

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1962. Min Act: -0.1700

Data Index: 1174407 (Open Web Text)

Max Activating Token Index: 79

Text #15

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1962. Min Act: -0.1700

Data Index: 1325114 (Open Web Text)

Max Activating Token Index: 445

Text #16

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.2068. Min Act: -0.1700

Data Index: 6106448 (Open Web Text)

Max Activating Token Index: 636

Text #17

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1669. Min Act: -0.1700

Data Index: 7845627 (Open Web Text)

Max Activating Token Index: 454

Text #18

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1475. Min Act: -0.1700

Data Index: 8421755 (Open Web Text)

Max Activating Token Index: 410

Text #19

Max Range: 3.3968. Min Range: -3.3968

Max Act: 3.1702. Min Act: -0.1700

Data Index: 3852087 (Open Web Text)

Max Activating Token Index: 241