Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3065 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.6247. Min Act: -0.0965

Data Index: 6161149 (Open Web Text)

Max Activating Token Index: 470

Text #1

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.6170. Min Act: -0.0574

Data Index: 1271979 (Open Web Text)

Max Activating Token Index: 577

Text #2

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5870. Min Act: -0.0917

Data Index: 3726631 (Open Web Text)

Max Activating Token Index: 483

Text #3

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5875. Min Act: -0.1240

Data Index: 1774262 (Open Web Text)

Max Activating Token Index: 345

Text #4

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5820. Min Act: -0.0868

Data Index: 7648679 (Open Web Text)

Max Activating Token Index: 627

Text #5

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5804. Min Act: -0.1025

Data Index: 4378063 (Open Web Text)

Max Activating Token Index: 529

Text #6

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5734. Min Act: -0.0750

Data Index: 6068154 (Open Web Text)

Max Activating Token Index: 598

Text #7

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5732. Min Act: -0.1491

Data Index: 2931281 (Open Web Text)

Max Activating Token Index: 715

Text #8

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5773. Min Act: -0.1013

Data Index: 651336 (Open Web Text)

Max Activating Token Index: 745

Text #9

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5735. Min Act: -0.0595

Data Index: 5254274 (Open Web Text)

Max Activating Token Index: 816

Text #10

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5641. Min Act: -0.0191

Data Index: 5823494 (Open Web Text)

Max Activating Token Index: 567

Text #11

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5682. Min Act: -0.1623

Data Index: 7966883 (Open Web Text)

Max Activating Token Index: 566

Text #12

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5682. Min Act: -0.0457

Data Index: 6981783 (Open Web Text)

Max Activating Token Index: 531

Text #13

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5665. Min Act: -0.1488

Data Index: 6375807 (Open Web Text)

Max Activating Token Index: 480

Text #14

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5701. Min Act: -0.1145

Data Index: 4515891 (Open Web Text)

Max Activating Token Index: 243

Text #15

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5641. Min Act: -0.0562

Data Index: 6841226 (Open Web Text)

Max Activating Token Index: 490

Text #16

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5644. Min Act: -0.1151

Data Index: 8816037 (Open Web Text)

Max Activating Token Index: 196

Text #17

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5622. Min Act: -0.0964

Data Index: 1870606 (Open Web Text)

Max Activating Token Index: 841

Text #18

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5560. Min Act: -0.0698

Data Index: 5247484 (Open Web Text)

Max Activating Token Index: 736

Text #19

Max Range: 1.6247. Min Range: -1.6247

Max Act: 1.5539. Min Act: -0.1206

Data Index: 4968367 (Open Web Text)

Max Activating Token Index: 634