Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1727 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0860. Min Range: -3.0860

Max Act: 3.0860. Min Act: -0.1700

Data Index: 5345519 (Open Web Text)

Max Activating Token Index: 195

Text #1

Max Range: 3.0860. Min Range: -3.0860

Max Act: 3.0592. Min Act: -0.1700

Data Index: 5240123 (Open Web Text)

Max Activating Token Index: 245

Text #2

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9909. Min Act: -0.1700

Data Index: 7010213 (Open Web Text)

Max Activating Token Index: 543

Text #3

Max Range: 3.0860. Min Range: -3.0860

Max Act: 3.0289. Min Act: -0.1700

Data Index: 3223451 (Open Web Text)

Max Activating Token Index: 810

Text #4

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9867. Min Act: -0.1700

Data Index: 3506807 (Open Web Text)

Max Activating Token Index: 49

Text #5

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9859. Min Act: -0.1700

Data Index: 5616761 (Open Web Text)

Max Activating Token Index: 190

Text #6

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9802. Min Act: -0.1700

Data Index: 8233586 (Open Web Text)

Max Activating Token Index: 569

Text #7

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9504. Min Act: -0.1700

Data Index: 3884366 (Open Web Text)

Max Activating Token Index: 33

Text #8

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9410. Min Act: -0.1700

Data Index: 5349949 (Open Web Text)

Max Activating Token Index: 173

Text #9

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9368. Min Act: -0.1700

Data Index: 5230032 (Open Web Text)

Max Activating Token Index: 835

Text #10

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9339. Min Act: -0.1700

Data Index: 3241179 (Open Web Text)

Max Activating Token Index: 488

Text #11

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9022. Min Act: -0.1700

Data Index: 8218875 (Open Web Text)

Max Activating Token Index: 904

Text #12

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9425. Min Act: -0.1700

Data Index: 1370156 (Open Web Text)

Max Activating Token Index: 870

Text #13

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9389. Min Act: -0.1700

Data Index: 3223661 (Open Web Text)

Max Activating Token Index: 767

Text #14

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9471. Min Act: -0.1700

Data Index: 1454763 (Open Web Text)

Max Activating Token Index: 36

Text #15

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9206. Min Act: -0.1700

Data Index: 6270734 (Open Web Text)

Max Activating Token Index: 457

Text #16

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9176. Min Act: -0.1700

Data Index: 7829017 (Open Web Text)

Max Activating Token Index: 611

Text #17

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9055. Min Act: -0.1700

Data Index: 1653869 (Open Web Text)

Max Activating Token Index: 117

Text #18

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9161. Min Act: -0.1700

Data Index: 8030708 (Open Web Text)

Max Activating Token Index: 577

Text #19

Max Range: 3.0860. Min Range: -3.0860

Max Act: 2.9103. Min Act: -0.1700

Data Index: 78980 (Open Web Text)

Max Activating Token Index: 706