Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 942 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.4496. Min Act: -0.1700

Data Index: 5987262 (Open Web Text)

Max Activating Token Index: 487

Text #1

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.4289. Min Act: -0.1700

Data Index: 1434659 (Open Web Text)

Max Activating Token Index: 803

Text #2

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3704. Min Act: -0.1700

Data Index: 6078925 (Open Web Text)

Max Activating Token Index: 522

Text #3

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3777. Min Act: -0.1700

Data Index: 1866035 (Open Web Text)

Max Activating Token Index: 443

Text #4

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3594. Min Act: -0.1700

Data Index: 2391357 (Open Web Text)

Max Activating Token Index: 40

Text #5

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3564. Min Act: -0.1700

Data Index: 5966033 (Open Web Text)

Max Activating Token Index: 550

Text #6

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.4001. Min Act: -0.1700

Data Index: 1231465 (Open Web Text)

Max Activating Token Index: 493

Text #7

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3590. Min Act: -0.1700

Data Index: 1717683 (Open Web Text)

Max Activating Token Index: 936

Text #8

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3630. Min Act: -0.1700

Data Index: 2851083 (Open Web Text)

Max Activating Token Index: 885

Text #9

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3310. Min Act: -0.1700

Data Index: 4610854 (Open Web Text)

Max Activating Token Index: 171

Text #10

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3400. Min Act: -0.1700

Data Index: 1605816 (Open Web Text)

Max Activating Token Index: 133

Text #11

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3264. Min Act: -0.1700

Data Index: 7220966 (Open Web Text)

Max Activating Token Index: 444

Text #12

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3175. Min Act: -0.1700

Data Index: 2602553 (Open Web Text)

Max Activating Token Index: 82

Text #13

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.2928. Min Act: -0.1700

Data Index: 4882038 (Open Web Text)

Max Activating Token Index: 180

Text #14

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.2765. Min Act: -0.1700

Data Index: 4893170 (Open Web Text)

Max Activating Token Index: 622

Text #15

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3185. Min Act: -0.1700

Data Index: 3421038 (Open Web Text)

Max Activating Token Index: 879

Text #16

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3238. Min Act: -0.1700

Data Index: 2976898 (Open Web Text)

Max Activating Token Index: 37

Text #17

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.2962. Min Act: -0.1700

Data Index: 4935414 (Open Web Text)

Max Activating Token Index: 570

Text #18

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.3298. Min Act: -0.1700

Data Index: 2602554 (Open Web Text)

Max Activating Token Index: 126

Text #19

Max Range: 4.4496. Min Range: -4.4496

Max Act: 4.2876. Min Act: -0.1700

Data Index: 1327151 (Open Web Text)

Max Activating Token Index: 54