Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 510 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.2298. Min Act: -0.1700

Data Index: 2223290 (Open Web Text)

Max Activating Token Index: 16

Text #1

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.1609. Min Act: -0.1700

Data Index: 4497136 (Open Web Text)

Max Activating Token Index: 29

Text #2

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.1749. Min Act: -0.1700

Data Index: 5423606 (Open Web Text)

Max Activating Token Index: 30

Text #3

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.1746. Min Act: -0.1700

Data Index: 6799805 (Open Web Text)

Max Activating Token Index: 17

Text #4

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.1208. Min Act: -0.1700

Data Index: 3731592 (Open Web Text)

Max Activating Token Index: 22

Text #5

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0767. Min Act: -0.1700

Data Index: 7207171 (Open Web Text)

Max Activating Token Index: 31

Text #6

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0944. Min Act: -0.1700

Data Index: 7418436 (Open Web Text)

Max Activating Token Index: 65

Text #7

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0612. Min Act: -0.1700

Data Index: 7418431 (Open Web Text)

Max Activating Token Index: 29

Text #8

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0448. Min Act: -0.1700

Data Index: 3626127 (Open Web Text)

Max Activating Token Index: 40

Text #9

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0622. Min Act: -0.1700

Data Index: 889381 (Open Web Text)

Max Activating Token Index: 32

Text #10

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0351. Min Act: -0.1700

Data Index: 8228164 (Open Web Text)

Max Activating Token Index: 26

Text #11

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0326. Min Act: -0.1700

Data Index: 8492883 (Open Web Text)

Max Activating Token Index: 17

Text #12

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0511. Min Act: -0.1700

Data Index: 7793731 (Open Web Text)

Max Activating Token Index: 25

Text #13

Max Range: 6.2298. Min Range: -6.2298

Max Act: 5.9933. Min Act: -0.1700

Data Index: 2364344 (Open Web Text)

Max Activating Token Index: 33

Text #14

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0002. Min Act: -0.1700

Data Index: 5333883 (Open Web Text)

Max Activating Token Index: 50

Text #15

Max Range: 6.2298. Min Range: -6.2298

Max Act: 5.9882. Min Act: -0.1700

Data Index: 7299009 (Open Web Text)

Max Activating Token Index: 7

Text #16

Max Range: 6.2298. Min Range: -6.2298

Max Act: 6.0270. Min Act: -0.1700

Data Index: 5521576 (Open Web Text)

Max Activating Token Index: 23

Text #17

Max Range: 6.2298. Min Range: -6.2298

Max Act: 5.9679. Min Act: -0.1700

Data Index: 6619001 (Open Web Text)

Max Activating Token Index: 30

Text #18

Max Range: 6.2298. Min Range: -6.2298

Max Act: 5.9543. Min Act: -0.1700

Data Index: 3008675 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 6.2298. Min Range: -6.2298

Max Act: 5.9509. Min Act: -0.1700

Data Index: 7418429 (Open Web Text)

Max Activating Token Index: 32