Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2782 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.6039. Min Act: -0.1142

Data Index: 2559247 (Open Web Text)

Max Activating Token Index: 561

Text #1

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5979. Min Act: -0.1246

Data Index: 6588571 (Open Web Text)

Max Activating Token Index: 415

Text #2

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5824. Min Act: -0.1142

Data Index: 5515109 (Open Web Text)

Max Activating Token Index: 491

Text #3

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5758. Min Act: -0.1145

Data Index: 4491974 (Open Web Text)

Max Activating Token Index: 392

Text #4

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5492. Min Act: -0.1160

Data Index: 868902 (Open Web Text)

Max Activating Token Index: 489

Text #5

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5550. Min Act: -0.1142

Data Index: 3524003 (Open Web Text)

Max Activating Token Index: 350

Text #6

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5524. Min Act: -0.1142

Data Index: 1429574 (Open Web Text)

Max Activating Token Index: 396

Text #7

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5451. Min Act: -0.1290

Data Index: 6652209 (Open Web Text)

Max Activating Token Index: 322

Text #8

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5473. Min Act: -0.1142

Data Index: 5822368 (Open Web Text)

Max Activating Token Index: 378

Text #9

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5487. Min Act: -0.1142

Data Index: 559977 (Open Web Text)

Max Activating Token Index: 561

Text #10

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5294. Min Act: -0.1554

Data Index: 3164463 (Open Web Text)

Max Activating Token Index: 512

Text #11

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5290. Min Act: -0.1176

Data Index: 8353439 (Open Web Text)

Max Activating Token Index: 577

Text #12

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5307. Min Act: -0.1180

Data Index: 3477841 (Open Web Text)

Max Activating Token Index: 758

Text #13

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5251. Min Act: -0.1290

Data Index: 6282086 (Open Web Text)

Max Activating Token Index: 421

Text #14

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5226. Min Act: -0.1159

Data Index: 3897613 (Open Web Text)

Max Activating Token Index: 902

Text #15

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5114. Min Act: -0.1425

Data Index: 2276619 (Open Web Text)

Max Activating Token Index: 475

Text #16

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5221. Min Act: -0.1165

Data Index: 420034 (Open Web Text)

Max Activating Token Index: 366

Text #17

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5063. Min Act: -0.1142

Data Index: 5484351 (Open Web Text)

Max Activating Token Index: 568

Text #18

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5247. Min Act: -0.1142

Data Index: 646021 (Open Web Text)

Max Activating Token Index: 895

Text #19

Max Range: 1.6039. Min Range: -1.6039

Max Act: 1.5164. Min Act: -0.1266

Data Index: 21679 (Open Web Text)

Max Activating Token Index: 436