Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1939 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0290. Min Range: -4.0290

Max Act: 4.0290. Min Act: -0.1700

Data Index: 382469 (Open Web Text)

Max Activating Token Index: 714

Text #1

Max Range: 4.0290. Min Range: -4.0290

Max Act: 4.0218. Min Act: -0.1700

Data Index: 4591792 (Open Web Text)

Max Activating Token Index: 239

Text #2

Max Range: 4.0290. Min Range: -4.0290

Max Act: 4.0123. Min Act: -0.1700

Data Index: 8174896 (Open Web Text)

Max Activating Token Index: 762

Text #3

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9773. Min Act: -0.1700

Data Index: 2806164 (Open Web Text)

Max Activating Token Index: 740

Text #4

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9728. Min Act: -0.1700

Data Index: 4597523 (Open Web Text)

Max Activating Token Index: 755

Text #5

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9326. Min Act: -0.1700

Data Index: 940894 (Open Web Text)

Max Activating Token Index: 736

Text #6

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9551. Min Act: -0.1700

Data Index: 3627630 (Open Web Text)

Max Activating Token Index: 717

Text #7

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9256. Min Act: -0.1700

Data Index: 6864320 (Open Web Text)

Max Activating Token Index: 575

Text #8

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9672. Min Act: -0.1700

Data Index: 7655495 (Open Web Text)

Max Activating Token Index: 1001

Text #9

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9407. Min Act: -0.1700

Data Index: 7780763 (Open Web Text)

Max Activating Token Index: 103

Text #10

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9039. Min Act: -0.1700

Data Index: 1950231 (Open Web Text)

Max Activating Token Index: 112

Text #11

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.9299. Min Act: -0.1700

Data Index: 1415745 (Open Web Text)

Max Activating Token Index: 584

Text #12

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8848. Min Act: -0.1700

Data Index: 8187781 (Open Web Text)

Max Activating Token Index: 882

Text #13

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8623. Min Act: -0.1700

Data Index: 6703650 (Open Web Text)

Max Activating Token Index: 536

Text #14

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8750. Min Act: -0.1700

Data Index: 6017433 (Open Web Text)

Max Activating Token Index: 114

Text #15

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8408. Min Act: -0.1700

Data Index: 3010096 (Open Web Text)

Max Activating Token Index: 664

Text #16

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8523. Min Act: -0.1700

Data Index: 2042232 (Open Web Text)

Max Activating Token Index: 653

Text #17

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8661. Min Act: -0.1700

Data Index: 281887 (Open Web Text)

Max Activating Token Index: 685

Text #18

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8277. Min Act: -0.1700

Data Index: 6343310 (Open Web Text)

Max Activating Token Index: 1011

Text #19

Max Range: 4.0290. Min Range: -4.0290

Max Act: 3.8286. Min Act: -0.1700

Data Index: 232872 (Open Web Text)

Max Activating Token Index: 175