Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2803 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.5061. Min Act: -0.0335

Data Index: 7711149 (Open Web Text)

Max Activating Token Index: 125

Text #1

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4873. Min Act: -0.1106

Data Index: 8652862 (Open Web Text)

Max Activating Token Index: 259

Text #2

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4853. Min Act: -0.0848

Data Index: 3922612 (Open Web Text)

Max Activating Token Index: 173

Text #3

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4837. Min Act: -0.0495

Data Index: 7450172 (Open Web Text)

Max Activating Token Index: 249

Text #4

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4819. Min Act: -0.0652

Data Index: 7647999 (Open Web Text)

Max Activating Token Index: 197

Text #5

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4690. Min Act: -0.0400

Data Index: 5633308 (Open Web Text)

Max Activating Token Index: 1022

Text #6

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4619. Min Act: -0.0412

Data Index: 313640 (Open Web Text)

Max Activating Token Index: 242

Text #7

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4574. Min Act: -0.0422

Data Index: 5567219 (Open Web Text)

Max Activating Token Index: 956

Text #8

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4585. Min Act: -0.0889

Data Index: 4734306 (Open Web Text)

Max Activating Token Index: 677

Text #9

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4645. Min Act: -0.0734

Data Index: 7059391 (Open Web Text)

Max Activating Token Index: 206

Text #10

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4605. Min Act: -0.0586

Data Index: 5129069 (Open Web Text)

Max Activating Token Index: 676

Text #11

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4636. Min Act: -0.0462

Data Index: 3878183 (Open Web Text)

Max Activating Token Index: 179

Text #12

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4523. Min Act: -0.0654

Data Index: 8153581 (Open Web Text)

Max Activating Token Index: 288

Text #13

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4577. Min Act: -0.0915

Data Index: 2480574 (Open Web Text)

Max Activating Token Index: 115

Text #14

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4609. Min Act: -0.0340

Data Index: 2400978 (Open Web Text)

Max Activating Token Index: 225

Text #15

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4580. Min Act: -0.0729

Data Index: 3537923 (Open Web Text)

Max Activating Token Index: 129

Text #16

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4530. Min Act: -0.0713

Data Index: 3417609 (Open Web Text)

Max Activating Token Index: 257

Text #17

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4543. Min Act: -0.0032

Data Index: 4429306 (Open Web Text)

Max Activating Token Index: 185

Text #18

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4557. Min Act: -0.0425

Data Index: 4401284 (Open Web Text)

Max Activating Token Index: 848

Text #19

Max Range: 1.5061. Min Range: -1.5061

Max Act: 1.4529. Min Act: -0.1137

Data Index: 4231862 (Open Web Text)

Max Activating Token Index: 201