Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 722 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.8383. Min Act: -0.1700

Data Index: 7251851 (Open Web Text)

Max Activating Token Index: 6

Text #1

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.7855. Min Act: -0.1700

Data Index: 3417209 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.8149. Min Act: -0.1700

Data Index: 5944440 (Open Web Text)

Max Activating Token Index: 11

Text #3

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6643. Min Act: -0.1700

Data Index: 8360095 (Open Web Text)

Max Activating Token Index: 9

Text #4

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6556. Min Act: -0.1700

Data Index: 8791007 (Open Web Text)

Max Activating Token Index: 10

Text #5

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6748. Min Act: -0.1700

Data Index: 5410829 (Open Web Text)

Max Activating Token Index: 18

Text #6

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6214. Min Act: -0.1700

Data Index: 7827031 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5989. Min Act: -0.1700

Data Index: 73782 (Open Web Text)

Max Activating Token Index: 17

Text #8

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6001. Min Act: -0.1700

Data Index: 3371964 (Open Web Text)

Max Activating Token Index: 286

Text #9

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6063. Min Act: -0.1700

Data Index: 8032392 (Open Web Text)

Max Activating Token Index: 17

Text #10

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6025. Min Act: -0.1700

Data Index: 3109738 (Open Web Text)

Max Activating Token Index: 569

Text #11

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6139. Min Act: -0.1700

Data Index: 5611010 (Open Web Text)

Max Activating Token Index: 10

Text #12

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.6142. Min Act: -0.1700

Data Index: 499159 (Open Web Text)

Max Activating Token Index: 12

Text #13

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5991. Min Act: -0.1700

Data Index: 7267724 (Open Web Text)

Max Activating Token Index: 275

Text #14

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5635. Min Act: -0.1700

Data Index: 2608999 (Open Web Text)

Max Activating Token Index: 14

Text #15

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5656. Min Act: -0.1700

Data Index: 528770 (Open Web Text)

Max Activating Token Index: 244

Text #16

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5527. Min Act: -0.1700

Data Index: 7926449 (Open Web Text)

Max Activating Token Index: 263

Text #17

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5712. Min Act: -0.1700

Data Index: 7624900 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5954. Min Act: -0.1700

Data Index: 5763602 (Open Web Text)

Max Activating Token Index: 18

Text #19

Max Range: 3.8383. Min Range: -3.8383

Max Act: 3.5620. Min Act: -0.1700

Data Index: 7562186 (Open Web Text)

Max Activating Token Index: 374