Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2568 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.1871. Min Act: -0.1700

Data Index: 7070277 (Open Web Text)

Max Activating Token Index: 762

Text #1

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.1852. Min Act: -0.1700

Data Index: 7978759 (Open Web Text)

Max Activating Token Index: 946

Text #2

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.1114. Min Act: -0.1700

Data Index: 7254852 (Open Web Text)

Max Activating Token Index: 43

Text #3

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.1282. Min Act: -0.1700

Data Index: 8449358 (Open Web Text)

Max Activating Token Index: 836

Text #4

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.1072. Min Act: -0.1700

Data Index: 6458315 (Open Web Text)

Max Activating Token Index: 917

Text #5

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0658. Min Act: -0.1700

Data Index: 6618989 (Open Web Text)

Max Activating Token Index: 357

Text #6

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0864. Min Act: -0.1700

Data Index: 8272992 (Open Web Text)

Max Activating Token Index: 266

Text #7

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0352. Min Act: -0.1700

Data Index: 3275688 (Open Web Text)

Max Activating Token Index: 71

Text #8

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0182. Min Act: -0.1700

Data Index: 519788 (Open Web Text)

Max Activating Token Index: 308

Text #9

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0147. Min Act: -0.1700

Data Index: 4086256 (Open Web Text)

Max Activating Token Index: 524

Text #10

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9426. Min Act: -0.1700

Data Index: 6126497 (Open Web Text)

Max Activating Token Index: 341

Text #11

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9748. Min Act: -0.1700

Data Index: 5153614 (Open Web Text)

Max Activating Token Index: 574

Text #12

Max Range: 5.1871. Min Range: -5.1871

Max Act: 5.0206. Min Act: -0.1700

Data Index: 3254778 (Open Web Text)

Max Activating Token Index: 571

Text #13

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9394. Min Act: -0.1700

Data Index: 637643 (Open Web Text)

Max Activating Token Index: 146

Text #14

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9218. Min Act: -0.1700

Data Index: 3451436 (Open Web Text)

Max Activating Token Index: 378

Text #15

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9182. Min Act: -0.1700

Data Index: 6122627 (Open Web Text)

Max Activating Token Index: 650

Text #16

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9353. Min Act: -0.1700

Data Index: 1061596 (Open Web Text)

Max Activating Token Index: 947

Text #17

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.8872. Min Act: -0.1700

Data Index: 7588231 (Open Web Text)

Max Activating Token Index: 514

Text #18

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.9038. Min Act: -0.1700

Data Index: 7662126 (Open Web Text)

Max Activating Token Index: 573

Text #19

Max Range: 5.1871. Min Range: -5.1871

Max Act: 4.8796. Min Act: -0.1700

Data Index: 5174017 (Open Web Text)

Max Activating Token Index: 120