Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1109 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.4417. Min Act: -0.1700

Data Index: 2945259 (Open Web Text)

Max Activating Token Index: 113

Text #1

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.4558. Min Act: -0.1700

Data Index: 4560907 (Open Web Text)

Max Activating Token Index: 996

Text #2

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3970. Min Act: -0.1700

Data Index: 4655732 (Open Web Text)

Max Activating Token Index: 338

Text #3

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.4078. Min Act: -0.1700

Data Index: 5285624 (Open Web Text)

Max Activating Token Index: 247

Text #4

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3493. Min Act: -0.1700

Data Index: 7086570 (Open Web Text)

Max Activating Token Index: 215

Text #5

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3480. Min Act: -0.1700

Data Index: 4602803 (Open Web Text)

Max Activating Token Index: 139

Text #6

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3464. Min Act: -0.1700

Data Index: 1079538 (Open Web Text)

Max Activating Token Index: 253

Text #7

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3389. Min Act: -0.1700

Data Index: 6671461 (Open Web Text)

Max Activating Token Index: 418

Text #8

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3582. Min Act: -0.1700

Data Index: 2369419 (Open Web Text)

Max Activating Token Index: 871

Text #9

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3087. Min Act: -0.1700

Data Index: 2260645 (Open Web Text)

Max Activating Token Index: 325

Text #10

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3094. Min Act: -0.1700

Data Index: 894248 (Open Web Text)

Max Activating Token Index: 151

Text #11

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.3259. Min Act: -0.1700

Data Index: 3136107 (Open Web Text)

Max Activating Token Index: 909

Text #12

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2849. Min Act: -0.1700

Data Index: 1002943 (Open Web Text)

Max Activating Token Index: 934

Text #13

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2499. Min Act: -0.1700

Data Index: 6011637 (Open Web Text)

Max Activating Token Index: 356

Text #14

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2578. Min Act: -0.1700

Data Index: 4193447 (Open Web Text)

Max Activating Token Index: 1009

Text #15

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2259. Min Act: -0.1700

Data Index: 8788691 (Open Web Text)

Max Activating Token Index: 674

Text #16

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2264. Min Act: -0.1700

Data Index: 7506319 (Open Web Text)

Max Activating Token Index: 362

Text #17

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2274. Min Act: -0.1700

Data Index: 4580541 (Open Web Text)

Max Activating Token Index: 63

Text #18

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.2254. Min Act: -0.1700

Data Index: 3775882 (Open Web Text)

Max Activating Token Index: 685

Text #19

Max Range: 4.4558. Min Range: -4.4558

Max Act: 4.1989. Min Act: -0.1700

Data Index: 3766477 (Open Web Text)

Max Activating Token Index: 303