Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2176 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0119. Min Range: -5.0119

Max Act: 5.0119. Min Act: -0.1700

Data Index: 5600181 (Open Web Text)

Max Activating Token Index: 1016

Text #1

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9716. Min Act: -0.1700

Data Index: 2722960 (Open Web Text)

Max Activating Token Index: 959

Text #2

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9660. Min Act: -0.1700

Data Index: 3571088 (Open Web Text)

Max Activating Token Index: 972

Text #3

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9859. Min Act: -0.1700

Data Index: 5697676 (Open Web Text)

Max Activating Token Index: 603

Text #4

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9717. Min Act: -0.1700

Data Index: 6006923 (Open Web Text)

Max Activating Token Index: 800

Text #5

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9688. Min Act: -0.1700

Data Index: 7696387 (Open Web Text)

Max Activating Token Index: 286

Text #6

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9390. Min Act: -0.1700

Data Index: 2667728 (Open Web Text)

Max Activating Token Index: 9

Text #7

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9464. Min Act: -0.1700

Data Index: 6746270 (Open Web Text)

Max Activating Token Index: 877

Text #8

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9414. Min Act: -0.1700

Data Index: 2625750 (Open Web Text)

Max Activating Token Index: 324

Text #9

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9251. Min Act: -0.1700

Data Index: 8694560 (Open Web Text)

Max Activating Token Index: 535

Text #10

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9465. Min Act: -0.1700

Data Index: 595410 (Open Web Text)

Max Activating Token Index: 209

Text #11

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9561. Min Act: -0.1700

Data Index: 6746278 (Open Web Text)

Max Activating Token Index: 468

Text #12

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9085. Min Act: -0.1700

Data Index: 4545660 (Open Web Text)

Max Activating Token Index: 283

Text #13

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8940. Min Act: -0.1700

Data Index: 6291722 (Open Web Text)

Max Activating Token Index: 176

Text #14

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8946. Min Act: -0.1700

Data Index: 2691603 (Open Web Text)

Max Activating Token Index: 19

Text #15

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8975. Min Act: -0.1700

Data Index: 6945856 (Open Web Text)

Max Activating Token Index: 243

Text #16

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8827. Min Act: -0.1700

Data Index: 1655587 (Open Web Text)

Max Activating Token Index: 809

Text #17

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8693. Min Act: -0.1700

Data Index: 6945857 (Open Web Text)

Max Activating Token Index: 853

Text #18

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.9062. Min Act: -0.1700

Data Index: 6746286 (Open Web Text)

Max Activating Token Index: 544

Text #19

Max Range: 5.0119. Min Range: -5.0119

Max Act: 4.8492. Min Act: -0.1700

Data Index: 595407 (Open Web Text)

Max Activating Token Index: 652