Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2426 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0354. Min Range: -4.0354

Max Act: 4.0354. Min Act: -0.1700

Data Index: 1489825 (Open Web Text)

Max Activating Token Index: 494

Text #1

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.9617. Min Act: -0.1700

Data Index: 234097 (Open Web Text)

Max Activating Token Index: 366

Text #2

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.9766. Min Act: -0.1700

Data Index: 8250700 (Open Web Text)

Max Activating Token Index: 523

Text #3

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.9118. Min Act: -0.1700

Data Index: 1926335 (Open Web Text)

Max Activating Token Index: 277

Text #4

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.9087. Min Act: -0.1700

Data Index: 8680827 (Open Web Text)

Max Activating Token Index: 513

Text #5

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.9004. Min Act: -0.1700

Data Index: 5298713 (Open Web Text)

Max Activating Token Index: 413

Text #6

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8910. Min Act: -0.1700

Data Index: 2732692 (Open Web Text)

Max Activating Token Index: 1018

Text #7

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8971. Min Act: -0.1700

Data Index: 6605708 (Open Web Text)

Max Activating Token Index: 655

Text #8

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8523. Min Act: -0.1700

Data Index: 319838 (Open Web Text)

Max Activating Token Index: 592

Text #9

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8373. Min Act: -0.1700

Data Index: 8575302 (Open Web Text)

Max Activating Token Index: 676

Text #10

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8729. Min Act: -0.1700

Data Index: 3364225 (Open Web Text)

Max Activating Token Index: 603

Text #11

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8530. Min Act: -0.1700

Data Index: 4866123 (Open Web Text)

Max Activating Token Index: 371

Text #12

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8645. Min Act: -0.1700

Data Index: 4798627 (Open Web Text)

Max Activating Token Index: 568

Text #13

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8648. Min Act: -0.1700

Data Index: 2005404 (Open Web Text)

Max Activating Token Index: 887

Text #14

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8492. Min Act: -0.1700

Data Index: 4154880 (Open Web Text)

Max Activating Token Index: 544

Text #15

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8771. Min Act: -0.1700

Data Index: 4758028 (Open Web Text)

Max Activating Token Index: 516

Text #16

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8767. Min Act: -0.1700

Data Index: 1282127 (Open Web Text)

Max Activating Token Index: 503

Text #17

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8137. Min Act: -0.1700

Data Index: 2584637 (Open Web Text)

Max Activating Token Index: 704

Text #18

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8636. Min Act: -0.1700

Data Index: 694838 (Open Web Text)

Max Activating Token Index: 1014

Text #19

Max Range: 4.0354. Min Range: -4.0354

Max Act: 3.8485. Min Act: -0.1700

Data Index: 7249207 (Open Web Text)

Max Activating Token Index: 956