Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2140 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1378. Min Range: -5.1378

Max Act: 5.1378. Min Act: -0.1700

Data Index: 3605491 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 5.1378. Min Range: -5.1378

Max Act: 5.0933. Min Act: -0.1700

Data Index: 2614392 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 5.1378. Min Range: -5.1378

Max Act: 5.0226. Min Act: -0.1700

Data Index: 5479388 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 5.1378. Min Range: -5.1378

Max Act: 5.0226. Min Act: -0.1700

Data Index: 3920559 (Open Web Text)

Max Activating Token Index: 2

Text #4

Max Range: 5.1378. Min Range: -5.1378

Max Act: 5.0215. Min Act: -0.1700

Data Index: 5716556 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9771. Min Act: -0.1700

Data Index: 2466925 (Open Web Text)

Max Activating Token Index: 31

Text #6

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9949. Min Act: -0.1700

Data Index: 1675166 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9563. Min Act: -0.1700

Data Index: 8037593 (Open Web Text)

Max Activating Token Index: 711

Text #8

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9876. Min Act: -0.1700

Data Index: 4136994 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9574. Min Act: -0.1700

Data Index: 3956727 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9584. Min Act: -0.1700

Data Index: 5624623 (Open Web Text)

Max Activating Token Index: 10

Text #11

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9442. Min Act: -0.1700

Data Index: 7294953 (Open Web Text)

Max Activating Token Index: 940

Text #12

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9339. Min Act: -0.1700

Data Index: 7046477 (Open Web Text)

Max Activating Token Index: 5

Text #13

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9263. Min Act: -0.1700

Data Index: 3997857 (Open Web Text)

Max Activating Token Index: 25

Text #14

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9217. Min Act: -0.1700

Data Index: 5810646 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9183. Min Act: -0.1700

Data Index: 4017904 (Open Web Text)

Max Activating Token Index: 16

Text #16

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9104. Min Act: -0.1700

Data Index: 5519544 (Open Web Text)

Max Activating Token Index: 15

Text #17

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.8911. Min Act: -0.1700

Data Index: 3202838 (Open Web Text)

Max Activating Token Index: 22

Text #18

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.8873. Min Act: -0.1700

Data Index: 1002403 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 5.1378. Min Range: -5.1378

Max Act: 4.9000. Min Act: -0.1700

Data Index: 1024196 (Open Web Text)

Max Activating Token Index: 33