Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1273 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1181. Min Range: -4.1181

Max Act: 4.1181. Min Act: -0.1700

Data Index: 1044419 (Open Web Text)

Max Activating Token Index: 806

Text #1

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.9208. Min Act: -0.1700

Data Index: 3256170 (Open Web Text)

Max Activating Token Index: 253

Text #2

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.9097. Min Act: -0.1700

Data Index: 7495226 (Open Web Text)

Max Activating Token Index: 362

Text #3

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8999. Min Act: -0.1700

Data Index: 3153135 (Open Web Text)

Max Activating Token Index: 475

Text #4

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8605. Min Act: -0.1700

Data Index: 7110377 (Open Web Text)

Max Activating Token Index: 985

Text #5

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8599. Min Act: -0.1700

Data Index: 7418308 (Open Web Text)

Max Activating Token Index: 735

Text #6

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8199. Min Act: -0.1700

Data Index: 7975759 (Open Web Text)

Max Activating Token Index: 948

Text #7

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8096. Min Act: -0.1700

Data Index: 3685587 (Open Web Text)

Max Activating Token Index: 883

Text #8

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8024. Min Act: -0.1700

Data Index: 3777097 (Open Web Text)

Max Activating Token Index: 983

Text #9

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7887. Min Act: -0.1700

Data Index: 5889589 (Open Web Text)

Max Activating Token Index: 114

Text #10

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.8070. Min Act: -0.1700

Data Index: 5889592 (Open Web Text)

Max Activating Token Index: 94

Text #11

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7585. Min Act: -0.1700

Data Index: 2731223 (Open Web Text)

Max Activating Token Index: 43

Text #12

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7812. Min Act: -0.1700

Data Index: 4123301 (Open Web Text)

Max Activating Token Index: 278

Text #13

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7785. Min Act: -0.1700

Data Index: 432336 (Open Web Text)

Max Activating Token Index: 87

Text #14

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7810. Min Act: -0.1700

Data Index: 5584967 (Open Web Text)

Max Activating Token Index: 987

Text #15

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7355. Min Act: -0.1700

Data Index: 3221767 (Open Web Text)

Max Activating Token Index: 643

Text #16

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7912. Min Act: -0.1700

Data Index: 5232143 (Open Web Text)

Max Activating Token Index: 949

Text #17

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7540. Min Act: -0.1700

Data Index: 1781723 (Open Web Text)

Max Activating Token Index: 539

Text #18

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7370. Min Act: -0.1700

Data Index: 1065680 (Open Web Text)

Max Activating Token Index: 387

Text #19

Max Range: 4.1181. Min Range: -4.1181

Max Act: 3.7673. Min Act: -0.1700

Data Index: 5596064 (Open Web Text)

Max Activating Token Index: 367