Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1231 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.9353. Min Act: -0.1700

Data Index: 2794798 (Open Web Text)

Max Activating Token Index: 22

Text #1

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.9150. Min Act: -0.1700

Data Index: 3228647 (Open Web Text)

Max Activating Token Index: 491

Text #2

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.9024. Min Act: -0.1700

Data Index: 3298815 (Open Web Text)

Max Activating Token Index: 846

Text #3

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.9080. Min Act: -0.1700

Data Index: 2139947 (Open Web Text)

Max Activating Token Index: 202

Text #4

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8971. Min Act: -0.1700

Data Index: 5039170 (Open Web Text)

Max Activating Token Index: 256

Text #5

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8767. Min Act: -0.1700

Data Index: 8208678 (Open Web Text)

Max Activating Token Index: 38

Text #6

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8757. Min Act: -0.1700

Data Index: 2989127 (Open Web Text)

Max Activating Token Index: 569

Text #7

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8701. Min Act: -0.1700

Data Index: 1621204 (Open Web Text)

Max Activating Token Index: 46

Text #8

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8600. Min Act: -0.1700

Data Index: 2939587 (Open Web Text)

Max Activating Token Index: 116

Text #9

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8631. Min Act: -0.1700

Data Index: 4089801 (Open Web Text)

Max Activating Token Index: 513

Text #10

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8740. Min Act: -0.1700

Data Index: 1241672 (Open Web Text)

Max Activating Token Index: 46

Text #11

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8460. Min Act: -0.1700

Data Index: 8036036 (Open Web Text)

Max Activating Token Index: 53

Text #12

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8543. Min Act: -0.1700

Data Index: 1763496 (Open Web Text)

Max Activating Token Index: 237

Text #13

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8639. Min Act: -0.1700

Data Index: 533274 (Open Web Text)

Max Activating Token Index: 268

Text #14

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8518. Min Act: -0.1700

Data Index: 7114337 (Open Web Text)

Max Activating Token Index: 375

Text #15

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8537. Min Act: -0.1700

Data Index: 5950955 (Open Web Text)

Max Activating Token Index: 472

Text #16

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8624. Min Act: -0.1700

Data Index: 457430 (Open Web Text)

Max Activating Token Index: 704

Text #17

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8462. Min Act: -0.1700

Data Index: 5006853 (Open Web Text)

Max Activating Token Index: 51

Text #18

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8370. Min Act: -0.1700

Data Index: 4013728 (Open Web Text)

Max Activating Token Index: 115

Text #19

Max Range: 1.9353. Min Range: -1.9353

Max Act: 1.8526. Min Act: -0.1700

Data Index: 6032 (Open Web Text)

Max Activating Token Index: 59