Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3061 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.2066. Min Act: -0.1700

Data Index: 3758354 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1928. Min Act: -0.1700

Data Index: 1165685 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1843. Min Act: -0.1700

Data Index: 5267652 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1541. Min Act: -0.1700

Data Index: 2603468 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1583. Min Act: -0.1700

Data Index: 7965097 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1303. Min Act: -0.1700

Data Index: 7956358 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1526. Min Act: -0.1700

Data Index: 7893482 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1618. Min Act: -0.1700

Data Index: 6094743 (Open Web Text)

Max Activating Token Index: 5

Text #8

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1444. Min Act: -0.1700

Data Index: 5716885 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1730. Min Act: -0.1700

Data Index: 372297 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1653. Min Act: -0.1700

Data Index: 5192488 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1423. Min Act: -0.1700

Data Index: 1583763 (Open Web Text)

Max Activating Token Index: 2

Text #12

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1674. Min Act: -0.1700

Data Index: 8167622 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1297. Min Act: -0.1700

Data Index: 1255551 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1306. Min Act: -0.1700

Data Index: 4992944 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1186. Min Act: -0.1700

Data Index: 610375 (Open Web Text)

Max Activating Token Index: 4

Text #16

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1339. Min Act: -0.1700

Data Index: 4082945 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1090. Min Act: -0.1700

Data Index: 3985648 (Open Web Text)

Max Activating Token Index: 3

Text #18

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1084. Min Act: -0.1700

Data Index: 6570069 (Open Web Text)

Max Activating Token Index: 4

Text #19

Max Range: 4.2066. Min Range: -4.2066

Max Act: 4.1162. Min Act: -0.1700

Data Index: 3889973 (Open Web Text)

Max Activating Token Index: 6