Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 81 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.6411. Min Act: -0.1257

Data Index: 3444353 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.6025. Min Act: -0.0649

Data Index: 6604351 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5868. Min Act: 0.0077

Data Index: 234375 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5746. Min Act: -0.0918

Data Index: 3482373 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5694. Min Act: -0.0396

Data Index: 7183239 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5732. Min Act: -0.1503

Data Index: 7897115 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5376. Min Act: -0.0108

Data Index: 235302 (Open Web Text)

Max Activating Token Index: 1023

Text #7

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5327. Min Act: -0.0330

Data Index: 6807918 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5313. Min Act: -0.0390

Data Index: 6080454 (Open Web Text)

Max Activating Token Index: 1023

Text #9

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5327. Min Act: -0.0614

Data Index: 6465935 (Open Web Text)

Max Activating Token Index: 1023

Text #10

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5275. Min Act: -0.0864

Data Index: 5335745 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5218. Min Act: -0.1095

Data Index: 6223640 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5143. Min Act: -0.0030

Data Index: 1299276 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5157. Min Act: -0.0352

Data Index: 2340931 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5162. Min Act: -0.0712

Data Index: 7717238 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5037. Min Act: -0.0851

Data Index: 4581143 (Open Web Text)

Max Activating Token Index: 1023

Text #16

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5097. Min Act: -0.0431

Data Index: 3913299 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.4962. Min Act: -0.0308

Data Index: 6188305 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.4991. Min Act: -0.0156

Data Index: 6146229 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 1.6411. Min Range: -1.6411

Max Act: 1.5021. Min Act: -0.0501

Data Index: 6906184 (Open Web Text)

Max Activating Token Index: 1023