Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2237 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1696. Min Range: -4.1696

Max Act: 4.1696. Min Act: -0.1700

Data Index: 2097370 (Open Web Text)

Max Activating Token Index: 984

Text #1

Max Range: 4.1696. Min Range: -4.1696

Max Act: 4.0389. Min Act: -0.1700

Data Index: 7035672 (Open Web Text)

Max Activating Token Index: 346

Text #2

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.9425. Min Act: -0.1700

Data Index: 8818470 (Open Web Text)

Max Activating Token Index: 249

Text #3

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.9415. Min Act: -0.1700

Data Index: 3468342 (Open Web Text)

Max Activating Token Index: 602

Text #4

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.9467. Min Act: -0.1700

Data Index: 4331974 (Open Web Text)

Max Activating Token Index: 499

Text #5

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.9063. Min Act: -0.1700

Data Index: 7926454 (Open Web Text)

Max Activating Token Index: 297

Text #6

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8807. Min Act: -0.1700

Data Index: 966278 (Open Web Text)

Max Activating Token Index: 1015

Text #7

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.9108. Min Act: -0.1700

Data Index: 7792152 (Open Web Text)

Max Activating Token Index: 261

Text #8

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8723. Min Act: -0.1700

Data Index: 5186991 (Open Web Text)

Max Activating Token Index: 640

Text #9

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8706. Min Act: -0.1700

Data Index: 392013 (Open Web Text)

Max Activating Token Index: 267

Text #10

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8696. Min Act: -0.1700

Data Index: 3909124 (Open Web Text)

Max Activating Token Index: 431

Text #11

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8756. Min Act: -0.1700

Data Index: 701496 (Open Web Text)

Max Activating Token Index: 562

Text #12

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8183. Min Act: -0.1700

Data Index: 2664567 (Open Web Text)

Max Activating Token Index: 690

Text #13

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8371. Min Act: -0.1700

Data Index: 2599819 (Open Web Text)

Max Activating Token Index: 543

Text #14

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8486. Min Act: -0.1700

Data Index: 5582949 (Open Web Text)

Max Activating Token Index: 585

Text #15

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8174. Min Act: -0.1700

Data Index: 195551 (Open Web Text)

Max Activating Token Index: 643

Text #16

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8069. Min Act: -0.1700

Data Index: 941976 (Open Web Text)

Max Activating Token Index: 312

Text #17

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.7979. Min Act: -0.1700

Data Index: 548821 (Open Web Text)

Max Activating Token Index: 915

Text #18

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8122. Min Act: -0.1700

Data Index: 7657440 (Open Web Text)

Max Activating Token Index: 273

Text #19

Max Range: 4.1696. Min Range: -4.1696

Max Act: 3.8140. Min Act: -0.1700

Data Index: 4878270 (Open Web Text)

Max Activating Token Index: 554