Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 268 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.9233. Min Act: -0.1700

Data Index: 8760434 (Open Web Text)

Max Activating Token Index: 356

Text #1

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.7399. Min Act: -0.1700

Data Index: 2789476 (Open Web Text)

Max Activating Token Index: 325

Text #2

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6985. Min Act: -0.1700

Data Index: 541401 (Open Web Text)

Max Activating Token Index: 165

Text #3

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6579. Min Act: -0.1700

Data Index: 8119277 (Open Web Text)

Max Activating Token Index: 984

Text #4

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.7826. Min Act: -0.1700

Data Index: 3816676 (Open Web Text)

Max Activating Token Index: 1023

Text #5

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6842. Min Act: -0.1700

Data Index: 6783348 (Open Web Text)

Max Activating Token Index: 526

Text #6

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6561. Min Act: -0.1700

Data Index: 8316496 (Open Web Text)

Max Activating Token Index: 235

Text #7

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6633. Min Act: -0.1700

Data Index: 649181 (Open Web Text)

Max Activating Token Index: 395

Text #8

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6399. Min Act: -0.1700

Data Index: 1405594 (Open Web Text)

Max Activating Token Index: 949

Text #9

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6288. Min Act: -0.1700

Data Index: 4412167 (Open Web Text)

Max Activating Token Index: 198

Text #10

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6149. Min Act: -0.1700

Data Index: 3188238 (Open Web Text)

Max Activating Token Index: 128

Text #11

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6418. Min Act: -0.1700

Data Index: 7803995 (Open Web Text)

Max Activating Token Index: 629

Text #12

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5383. Min Act: -0.1700

Data Index: 4343010 (Open Web Text)

Max Activating Token Index: 401

Text #13

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5919. Min Act: -0.1700

Data Index: 7218840 (Open Web Text)

Max Activating Token Index: 135

Text #14

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.6052. Min Act: -0.1700

Data Index: 7219216 (Open Web Text)

Max Activating Token Index: 101

Text #15

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5369. Min Act: -0.1700

Data Index: 3153500 (Open Web Text)

Max Activating Token Index: 870

Text #16

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5677. Min Act: -0.1700

Data Index: 7629123 (Open Web Text)

Max Activating Token Index: 655

Text #17

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5386. Min Act: -0.1700

Data Index: 3197805 (Open Web Text)

Max Activating Token Index: 196

Text #18

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5368. Min Act: -0.1700

Data Index: 1915975 (Open Web Text)

Max Activating Token Index: 384

Text #19

Max Range: 3.9233. Min Range: -3.9233

Max Act: 3.5378. Min Act: -0.1700

Data Index: 730438 (Open Web Text)

Max Activating Token Index: 852