Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2069 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.8341. Min Act: -0.1700

Data Index: 2459954 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7967. Min Act: -0.1700

Data Index: 4143318 (Open Web Text)

Max Activating Token Index: 715

Text #2

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7920. Min Act: -0.1700

Data Index: 3872377 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7651. Min Act: -0.1700

Data Index: 327226 (Open Web Text)

Max Activating Token Index: 103

Text #4

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7511. Min Act: -0.1700

Data Index: 1741868 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7602. Min Act: -0.1700

Data Index: 3290445 (Open Web Text)

Max Activating Token Index: 10

Text #6

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7351. Min Act: -0.1700

Data Index: 310537 (Open Web Text)

Max Activating Token Index: 969

Text #7

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7367. Min Act: -0.1700

Data Index: 6365034 (Open Web Text)

Max Activating Token Index: 956

Text #8

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7038. Min Act: -0.1700

Data Index: 3998748 (Open Web Text)

Max Activating Token Index: 1015

Text #9

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7412. Min Act: -0.1700

Data Index: 556266 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7224. Min Act: -0.1700

Data Index: 1451570 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7205. Min Act: -0.1700

Data Index: 1789685 (Open Web Text)

Max Activating Token Index: 6

Text #12

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6961. Min Act: -0.1700

Data Index: 7772746 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6953. Min Act: -0.1700

Data Index: 2137819 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6963. Min Act: -0.1700

Data Index: 2446778 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.7109. Min Act: -0.1700

Data Index: 960714 (Open Web Text)

Max Activating Token Index: 480

Text #16

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6610. Min Act: -0.1700

Data Index: 2960268 (Open Web Text)

Max Activating Token Index: 149

Text #17

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6547. Min Act: -0.1700

Data Index: 4992773 (Open Web Text)

Max Activating Token Index: 7

Text #18

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6805. Min Act: -0.1700

Data Index: 179810 (Open Web Text)

Max Activating Token Index: 10

Text #19

Max Range: 4.8341. Min Range: -4.8341

Max Act: 4.6670. Min Act: -0.1700

Data Index: 1829479 (Open Web Text)

Max Activating Token Index: 3