Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 265 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.6077. Min Act: -0.1700

Data Index: 381911 (Open Web Text)

Max Activating Token Index: 197

Text #1

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5660. Min Act: -0.1700

Data Index: 5693878 (Open Web Text)

Max Activating Token Index: 682

Text #2

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5517. Min Act: -0.1700

Data Index: 902572 (Open Web Text)

Max Activating Token Index: 213

Text #3

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5260. Min Act: -0.1700

Data Index: 6515616 (Open Web Text)

Max Activating Token Index: 521

Text #4

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5131. Min Act: -0.1700

Data Index: 3283190 (Open Web Text)

Max Activating Token Index: 731

Text #5

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5031. Min Act: -0.1700

Data Index: 4537008 (Open Web Text)

Max Activating Token Index: 164

Text #6

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.5042. Min Act: -0.1700

Data Index: 5432300 (Open Web Text)

Max Activating Token Index: 457

Text #7

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4798. Min Act: -0.1700

Data Index: 5063939 (Open Web Text)

Max Activating Token Index: 481

Text #8

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4458. Min Act: -0.1700

Data Index: 831586 (Open Web Text)

Max Activating Token Index: 446

Text #9

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4442. Min Act: -0.1700

Data Index: 1684108 (Open Web Text)

Max Activating Token Index: 210

Text #10

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4447. Min Act: -0.1700

Data Index: 1554764 (Open Web Text)

Max Activating Token Index: 563

Text #11

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4327. Min Act: -0.1700

Data Index: 274678 (Open Web Text)

Max Activating Token Index: 63

Text #12

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4348. Min Act: -0.1700

Data Index: 3142599 (Open Web Text)

Max Activating Token Index: 720

Text #13

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4347. Min Act: -0.1700

Data Index: 237346 (Open Web Text)

Max Activating Token Index: 424

Text #14

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4561. Min Act: -0.1700

Data Index: 6778606 (Open Web Text)

Max Activating Token Index: 320

Text #15

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4489. Min Act: -0.1700

Data Index: 4724684 (Open Web Text)

Max Activating Token Index: 69

Text #16

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4516. Min Act: -0.1700

Data Index: 7240064 (Open Web Text)

Max Activating Token Index: 433

Text #17

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4010. Min Act: -0.1700

Data Index: 6895405 (Open Web Text)

Max Activating Token Index: 199

Text #18

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.4109. Min Act: -0.1700

Data Index: 6942433 (Open Web Text)

Max Activating Token Index: 508

Text #19

Max Range: 4.6077. Min Range: -4.6077

Max Act: 4.3977. Min Act: -0.1700

Data Index: 6632249 (Open Web Text)

Max Activating Token Index: 644