Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2902 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.8103. Min Act: -0.1700

Data Index: 8555973 (Open Web Text)

Max Activating Token Index: 847

Text #1

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.7988. Min Act: -0.1700

Data Index: 4509444 (Open Web Text)

Max Activating Token Index: 604

Text #2

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.7587. Min Act: -0.1700

Data Index: 1315580 (Open Web Text)

Max Activating Token Index: 675

Text #3

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.7358. Min Act: -0.1700

Data Index: 4185999 (Open Web Text)

Max Activating Token Index: 525

Text #4

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.7413. Min Act: -0.1700

Data Index: 750361 (Open Web Text)

Max Activating Token Index: 408

Text #5

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6721. Min Act: -0.1700

Data Index: 5911696 (Open Web Text)

Max Activating Token Index: 680

Text #6

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6909. Min Act: -0.1700

Data Index: 3922184 (Open Web Text)

Max Activating Token Index: 379

Text #7

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6639. Min Act: -0.1700

Data Index: 8482727 (Open Web Text)

Max Activating Token Index: 34

Text #8

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6936. Min Act: -0.1700

Data Index: 5993805 (Open Web Text)

Max Activating Token Index: 754

Text #9

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6409. Min Act: -0.1700

Data Index: 5986646 (Open Web Text)

Max Activating Token Index: 618

Text #10

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6481. Min Act: -0.1700

Data Index: 1817265 (Open Web Text)

Max Activating Token Index: 1002

Text #11

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6385. Min Act: -0.1700

Data Index: 3061297 (Open Web Text)

Max Activating Token Index: 556

Text #12

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6115. Min Act: -0.1700

Data Index: 6190400 (Open Web Text)

Max Activating Token Index: 313

Text #13

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6222. Min Act: -0.1700

Data Index: 5915724 (Open Web Text)

Max Activating Token Index: 54

Text #14

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6135. Min Act: -0.1700

Data Index: 7122623 (Open Web Text)

Max Activating Token Index: 579

Text #15

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6190. Min Act: -0.1700

Data Index: 8396306 (Open Web Text)

Max Activating Token Index: 580

Text #16

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.6023. Min Act: -0.1700

Data Index: 7086525 (Open Web Text)

Max Activating Token Index: 975

Text #17

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.5746. Min Act: -0.1700

Data Index: 4781627 (Open Web Text)

Max Activating Token Index: 930

Text #18

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.5783. Min Act: -0.1700

Data Index: 5331223 (Open Web Text)

Max Activating Token Index: 600

Text #19

Max Range: 4.8103. Min Range: -4.8103

Max Act: 4.5867. Min Act: -0.1700

Data Index: 7246642 (Open Web Text)

Max Activating Token Index: 596