Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2279 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0848. Min Act: -0.1700

Data Index: 2076335 (Open Web Text)

Max Activating Token Index: 702

Text #1

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0645. Min Act: -0.1700

Data Index: 5364347 (Open Web Text)

Max Activating Token Index: 451

Text #2

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0114. Min Act: -0.1700

Data Index: 3522520 (Open Web Text)

Max Activating Token Index: 694

Text #3

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0179. Min Act: -0.1700

Data Index: 6138610 (Open Web Text)

Max Activating Token Index: 667

Text #4

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0129. Min Act: -0.1700

Data Index: 7587968 (Open Web Text)

Max Activating Token Index: 484

Text #5

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0210. Min Act: -0.1700

Data Index: 3757806 (Open Web Text)

Max Activating Token Index: 860

Text #6

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9902. Min Act: -0.1700

Data Index: 8442167 (Open Web Text)

Max Activating Token Index: 398

Text #7

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9791. Min Act: -0.1700

Data Index: 1141418 (Open Web Text)

Max Activating Token Index: 457

Text #8

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9580. Min Act: -0.1700

Data Index: 4485671 (Open Web Text)

Max Activating Token Index: 800

Text #9

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9752. Min Act: -0.1700

Data Index: 1350375 (Open Web Text)

Max Activating Token Index: 919

Text #10

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9980. Min Act: -0.1700

Data Index: 2147095 (Open Web Text)

Max Activating Token Index: 1013

Text #11

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0342. Min Act: -0.1700

Data Index: 2301387 (Open Web Text)

Max Activating Token Index: 482

Text #12

Max Range: 4.0848. Min Range: -4.0848

Max Act: 4.0205. Min Act: -0.1700

Data Index: 1056792 (Open Web Text)

Max Activating Token Index: 178

Text #13

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9255. Min Act: -0.1700

Data Index: 3972520 (Open Web Text)

Max Activating Token Index: 396

Text #14

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9632. Min Act: -0.1700

Data Index: 6792898 (Open Web Text)

Max Activating Token Index: 115

Text #15

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9635. Min Act: -0.1700

Data Index: 8075327 (Open Web Text)

Max Activating Token Index: 301

Text #16

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9912. Min Act: -0.1700

Data Index: 5601389 (Open Web Text)

Max Activating Token Index: 977

Text #17

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9590. Min Act: -0.1700

Data Index: 4354612 (Open Web Text)

Max Activating Token Index: 198

Text #18

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9504. Min Act: -0.1700

Data Index: 7819420 (Open Web Text)

Max Activating Token Index: 286

Text #19

Max Range: 4.0848. Min Range: -4.0848

Max Act: 3.9232. Min Act: -0.1700

Data Index: 5553462 (Open Web Text)

Max Activating Token Index: 102