Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2167 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.9378. Min Act: -0.1700

Data Index: 2816688 (Open Web Text)

Max Activating Token Index: 454

Text #1

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.8705. Min Act: -0.1700

Data Index: 8037974 (Open Web Text)

Max Activating Token Index: 508

Text #2

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.9150. Min Act: -0.1700

Data Index: 3275618 (Open Web Text)

Max Activating Token Index: 71

Text #3

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7994. Min Act: -0.1700

Data Index: 4876874 (Open Web Text)

Max Activating Token Index: 277

Text #4

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.8501. Min Act: -0.1700

Data Index: 8705339 (Open Web Text)

Max Activating Token Index: 506

Text #5

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7925. Min Act: -0.1700

Data Index: 7922106 (Open Web Text)

Max Activating Token Index: 125

Text #6

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.8313. Min Act: -0.1700

Data Index: 6196192 (Open Web Text)

Max Activating Token Index: 500

Text #7

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.8078. Min Act: -0.1700

Data Index: 4384276 (Open Web Text)

Max Activating Token Index: 931

Text #8

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7276. Min Act: -0.1700

Data Index: 8378515 (Open Web Text)

Max Activating Token Index: 220

Text #9

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7324. Min Act: -0.1700

Data Index: 2169165 (Open Web Text)

Max Activating Token Index: 739

Text #10

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7297. Min Act: -0.1700

Data Index: 5673079 (Open Web Text)

Max Activating Token Index: 157

Text #11

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7404. Min Act: -0.1700

Data Index: 1029531 (Open Web Text)

Max Activating Token Index: 352

Text #12

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7187. Min Act: -0.1700

Data Index: 1809260 (Open Web Text)

Max Activating Token Index: 58

Text #13

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6873. Min Act: -0.1700

Data Index: 4266379 (Open Web Text)

Max Activating Token Index: 387

Text #14

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.7343. Min Act: -0.1700

Data Index: 2654831 (Open Web Text)

Max Activating Token Index: 1010

Text #15

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6999. Min Act: -0.1700

Data Index: 1551459 (Open Web Text)

Max Activating Token Index: 430

Text #16

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6482. Min Act: -0.1700

Data Index: 5068394 (Open Web Text)

Max Activating Token Index: 152

Text #17

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6699. Min Act: -0.1700

Data Index: 1982271 (Open Web Text)

Max Activating Token Index: 264

Text #18

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6453. Min Act: -0.1700

Data Index: 7721379 (Open Web Text)

Max Activating Token Index: 387

Text #19

Max Range: 5.9378. Min Range: -5.9378

Max Act: 5.6561. Min Act: -0.1700

Data Index: 3599094 (Open Web Text)

Max Activating Token Index: 450