Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1819 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0338. Min Range: -4.0338

Max Act: 4.0338. Min Act: -0.1700

Data Index: 550272 (Open Web Text)

Max Activating Token Index: 333

Text #1

Max Range: 4.0338. Min Range: -4.0338

Max Act: 4.0006. Min Act: -0.1700

Data Index: 831696 (Open Web Text)

Max Activating Token Index: 428

Text #2

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9656. Min Act: -0.1700

Data Index: 3054410 (Open Web Text)

Max Activating Token Index: 411

Text #3

Max Range: 4.0338. Min Range: -4.0338

Max Act: 4.0114. Min Act: -0.1700

Data Index: 8056680 (Open Web Text)

Max Activating Token Index: 144

Text #4

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9963. Min Act: -0.1700

Data Index: 1044229 (Open Web Text)

Max Activating Token Index: 495

Text #5

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9745. Min Act: -0.1700

Data Index: 2246491 (Open Web Text)

Max Activating Token Index: 72

Text #6

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9386. Min Act: -0.1700

Data Index: 6170205 (Open Web Text)

Max Activating Token Index: 94

Text #7

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9234. Min Act: -0.1700

Data Index: 1091736 (Open Web Text)

Max Activating Token Index: 573

Text #8

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9528. Min Act: -0.1700

Data Index: 5871919 (Open Web Text)

Max Activating Token Index: 425

Text #9

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9418. Min Act: -0.1700

Data Index: 5481440 (Open Web Text)

Max Activating Token Index: 645

Text #10

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9507. Min Act: -0.1700

Data Index: 2485786 (Open Web Text)

Max Activating Token Index: 406

Text #11

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9041. Min Act: -0.1700

Data Index: 6109767 (Open Web Text)

Max Activating Token Index: 273

Text #12

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.8985. Min Act: -0.1700

Data Index: 5544784 (Open Web Text)

Max Activating Token Index: 92

Text #13

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9008. Min Act: -0.1700

Data Index: 135366 (Open Web Text)

Max Activating Token Index: 460

Text #14

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.8856. Min Act: -0.1700

Data Index: 1020954 (Open Web Text)

Max Activating Token Index: 472

Text #15

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.8781. Min Act: -0.1700

Data Index: 5511000 (Open Web Text)

Max Activating Token Index: 63

Text #16

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9023. Min Act: -0.1700

Data Index: 132220 (Open Web Text)

Max Activating Token Index: 1012

Text #17

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9124. Min Act: -0.1700

Data Index: 4503301 (Open Web Text)

Max Activating Token Index: 942

Text #18

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9055. Min Act: -0.1700

Data Index: 3647634 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 4.0338. Min Range: -4.0338

Max Act: 3.9297. Min Act: -0.1700

Data Index: 3125907 (Open Web Text)

Max Activating Token Index: 712