Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2961 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4287. Min Act: -0.1700

Data Index: 4478609 (Open Web Text)

Max Activating Token Index: 150

Text #1

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4351. Min Act: -0.1700

Data Index: 7043884 (Open Web Text)

Max Activating Token Index: 351

Text #2

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4168. Min Act: -0.1700

Data Index: 4926584 (Open Web Text)

Max Activating Token Index: 148

Text #3

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3985. Min Act: -0.1700

Data Index: 6656084 (Open Web Text)

Max Activating Token Index: 149

Text #4

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4191. Min Act: -0.1700

Data Index: 3377289 (Open Web Text)

Max Activating Token Index: 42

Text #5

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4215. Min Act: -0.1700

Data Index: 4108610 (Open Web Text)

Max Activating Token Index: 47

Text #6

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4215. Min Act: -0.1700

Data Index: 7549540 (Open Web Text)

Max Activating Token Index: 46

Text #7

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4163. Min Act: -0.1700

Data Index: 8187244 (Open Web Text)

Max Activating Token Index: 182

Text #8

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4038. Min Act: -0.1700

Data Index: 6345993 (Open Web Text)

Max Activating Token Index: 171

Text #9

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3986. Min Act: -0.1700

Data Index: 4373701 (Open Web Text)

Max Activating Token Index: 60

Text #10

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3966. Min Act: -0.1700

Data Index: 6089352 (Open Web Text)

Max Activating Token Index: 203

Text #11

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4130. Min Act: -0.1700

Data Index: 6909104 (Open Web Text)

Max Activating Token Index: 39

Text #12

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3742. Min Act: -0.1700

Data Index: 1872743 (Open Web Text)

Max Activating Token Index: 181

Text #13

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4029. Min Act: -0.1700

Data Index: 1913780 (Open Web Text)

Max Activating Token Index: 55

Text #14

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3861. Min Act: -0.1700

Data Index: 4505996 (Open Web Text)

Max Activating Token Index: 219

Text #15

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.4042. Min Act: -0.1700

Data Index: 4652383 (Open Web Text)

Max Activating Token Index: 153

Text #16

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3925. Min Act: -0.1700

Data Index: 4892451 (Open Web Text)

Max Activating Token Index: 209

Text #17

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3821. Min Act: -0.1700

Data Index: 1845762 (Open Web Text)

Max Activating Token Index: 179

Text #18

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3885. Min Act: -0.1700

Data Index: 5552564 (Open Web Text)

Max Activating Token Index: 120

Text #19

Max Range: 3.4351. Min Range: -3.4351

Max Act: 3.3795. Min Act: -0.1700

Data Index: 2277367 (Open Web Text)

Max Activating Token Index: 149