Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1059 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5143. Min Range: -4.5143

Max Act: 4.5143. Min Act: -0.1700

Data Index: 2514008 (Open Web Text)

Max Activating Token Index: 578

Text #1

Max Range: 4.5143. Min Range: -4.5143

Max Act: 4.0193. Min Act: -0.1700

Data Index: 1281279 (Open Web Text)

Max Activating Token Index: 394

Text #2

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.4317. Min Act: -0.1700

Data Index: 3637669 (Open Web Text)

Max Activating Token Index: 755

Text #3

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.4934. Min Act: -0.1700

Data Index: 7738091 (Open Web Text)

Max Activating Token Index: 573

Text #4

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.4631. Min Act: -0.1700

Data Index: 6290665 (Open Web Text)

Max Activating Token Index: 186

Text #5

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.3203. Min Act: -0.1700

Data Index: 5388484 (Open Web Text)

Max Activating Token Index: 285

Text #6

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.2796. Min Act: -0.1700

Data Index: 4721783 (Open Web Text)

Max Activating Token Index: 223

Text #7

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.3074. Min Act: -0.1700

Data Index: 6743463 (Open Web Text)

Max Activating Token Index: 83

Text #8

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.3118. Min Act: -0.1700

Data Index: 4450937 (Open Web Text)

Max Activating Token Index: 263

Text #9

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.2176. Min Act: -0.1700

Data Index: 8255585 (Open Web Text)

Max Activating Token Index: 55

Text #10

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.1393. Min Act: -0.1700

Data Index: 6881139 (Open Web Text)

Max Activating Token Index: 876

Text #11

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.1185. Min Act: -0.1700

Data Index: 1714899 (Open Web Text)

Max Activating Token Index: 189

Text #12

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.1979. Min Act: -0.1700

Data Index: 1064926 (Open Web Text)

Max Activating Token Index: 387

Text #13

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.0931. Min Act: -0.1700

Data Index: 215387 (Open Web Text)

Max Activating Token Index: 289

Text #14

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.0798. Min Act: -0.1700

Data Index: 669709 (Open Web Text)

Max Activating Token Index: 33

Text #15

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.1052. Min Act: -0.1700

Data Index: 5189675 (Open Web Text)

Max Activating Token Index: 933

Text #16

Max Range: 4.5143. Min Range: -4.5143

Max Act: 2.9556. Min Act: -0.1700

Data Index: 3585916 (Open Web Text)

Max Activating Token Index: 588

Text #17

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.0533. Min Act: -0.1700

Data Index: 5258004 (Open Web Text)

Max Activating Token Index: 17

Text #18

Max Range: 4.5143. Min Range: -4.5143

Max Act: 3.0217. Min Act: -0.1700

Data Index: 6807978 (Open Web Text)

Max Activating Token Index: 509

Text #19

Max Range: 4.5143. Min Range: -4.5143

Max Act: 2.9707. Min Act: -0.1700

Data Index: 554697 (Open Web Text)

Max Activating Token Index: 726