Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 862 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.7360. Min Act: -0.1700

Data Index: 1745733 (Open Web Text)

Max Activating Token Index: 421

Text #1

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6481. Min Act: -0.1700

Data Index: 3301283 (Open Web Text)

Max Activating Token Index: 593

Text #2

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6485. Min Act: -0.1700

Data Index: 7646727 (Open Web Text)

Max Activating Token Index: 1022

Text #3

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6124. Min Act: -0.1700

Data Index: 7603763 (Open Web Text)

Max Activating Token Index: 360

Text #4

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5863. Min Act: -0.1700

Data Index: 4582138 (Open Web Text)

Max Activating Token Index: 426

Text #5

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5549. Min Act: -0.1700

Data Index: 1532628 (Open Web Text)

Max Activating Token Index: 830

Text #6

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6183. Min Act: -0.1700

Data Index: 2280217 (Open Web Text)

Max Activating Token Index: 218

Text #7

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5588. Min Act: -0.1700

Data Index: 893201 (Open Web Text)

Max Activating Token Index: 578

Text #8

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5670. Min Act: -0.1700

Data Index: 6309452 (Open Web Text)

Max Activating Token Index: 746

Text #9

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5836. Min Act: -0.1700

Data Index: 8661479 (Open Web Text)

Max Activating Token Index: 638

Text #10

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6155. Min Act: -0.1700

Data Index: 3607644 (Open Web Text)

Max Activating Token Index: 388

Text #11

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.6044. Min Act: -0.1700

Data Index: 3531984 (Open Web Text)

Max Activating Token Index: 118

Text #12

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5761. Min Act: -0.1700

Data Index: 8415883 (Open Web Text)

Max Activating Token Index: 967

Text #13

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5551. Min Act: -0.1700

Data Index: 1267372 (Open Web Text)

Max Activating Token Index: 626

Text #14

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5597. Min Act: -0.1700

Data Index: 6041370 (Open Web Text)

Max Activating Token Index: 787

Text #15

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5345. Min Act: -0.1700

Data Index: 1030913 (Open Web Text)

Max Activating Token Index: 945

Text #16

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5553. Min Act: -0.1700

Data Index: 6311777 (Open Web Text)

Max Activating Token Index: 631

Text #17

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5905. Min Act: -0.1700

Data Index: 6697829 (Open Web Text)

Max Activating Token Index: 774

Text #18

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5055. Min Act: -0.1700

Data Index: 7474263 (Open Web Text)

Max Activating Token Index: 229

Text #19

Max Range: 3.7360. Min Range: -3.7360

Max Act: 3.5026. Min Act: -0.1700

Data Index: 5173570 (Open Web Text)

Max Activating Token Index: 857