Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1163 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.8370. Min Act: -0.1700

Data Index: 59531 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7933. Min Act: -0.1700

Data Index: 3007345 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7839. Min Act: -0.1700

Data Index: 271269 (Open Web Text)

Max Activating Token Index: 326

Text #3

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7579. Min Act: -0.1700

Data Index: 5680441 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7388. Min Act: -0.1700

Data Index: 1965266 (Open Web Text)

Max Activating Token Index: 8

Text #5

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7209. Min Act: -0.1700

Data Index: 6732794 (Open Web Text)

Max Activating Token Index: 15

Text #6

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7347. Min Act: -0.1700

Data Index: 4611954 (Open Web Text)

Max Activating Token Index: 947

Text #7

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7230. Min Act: -0.1700

Data Index: 8409206 (Open Web Text)

Max Activating Token Index: 53

Text #8

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7059. Min Act: -0.1700

Data Index: 6261141 (Open Web Text)

Max Activating Token Index: 134

Text #9

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7062. Min Act: -0.1700

Data Index: 5981567 (Open Web Text)

Max Activating Token Index: 9

Text #10

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6994. Min Act: -0.1700

Data Index: 5599781 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6934. Min Act: -0.1700

Data Index: 155617 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6903. Min Act: -0.1700

Data Index: 1106537 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7013. Min Act: -0.1700

Data Index: 2346637 (Open Web Text)

Max Activating Token Index: 144

Text #14

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6994. Min Act: -0.1700

Data Index: 7418447 (Open Web Text)

Max Activating Token Index: 702

Text #15

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.7018. Min Act: -0.1700

Data Index: 7861951 (Open Web Text)

Max Activating Token Index: 841

Text #16

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6845. Min Act: -0.1700

Data Index: 3963375 (Open Web Text)

Max Activating Token Index: 14

Text #17

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6796. Min Act: -0.1700

Data Index: 6921379 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6791. Min Act: -0.1700

Data Index: 2712567 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 2.8370. Min Range: -2.8370

Max Act: 2.6805. Min Act: -0.1700

Data Index: 1576511 (Open Web Text)

Max Activating Token Index: 14