Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1383 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.8467. Min Act: -0.1202

Data Index: 3063685 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.8417. Min Act: -0.1029

Data Index: 7593841 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.8023. Min Act: -0.1258

Data Index: 7983511 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7955. Min Act: -0.1055

Data Index: 6081276 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7890. Min Act: -0.1005

Data Index: 2173770 (Open Web Text)

Max Activating Token Index: 463

Text #5

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7744. Min Act: -0.0911

Data Index: 4706325 (Open Web Text)

Max Activating Token Index: 1023

Text #6

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7698. Min Act: -0.0988

Data Index: 6431188 (Open Web Text)

Max Activating Token Index: 481

Text #7

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7791. Min Act: -0.0395

Data Index: 2936541 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7638. Min Act: -0.1044

Data Index: 1180567 (Open Web Text)

Max Activating Token Index: 475

Text #9

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7686. Min Act: -0.1089

Data Index: 2252824 (Open Web Text)

Max Activating Token Index: 665

Text #10

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7632. Min Act: -0.1269

Data Index: 1669721 (Open Web Text)

Max Activating Token Index: 1023

Text #11

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7638. Min Act: -0.0999

Data Index: 6341516 (Open Web Text)

Max Activating Token Index: 1023

Text #12

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7615. Min Act: -0.1094

Data Index: 8182741 (Open Web Text)

Max Activating Token Index: 569

Text #13

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7668. Min Act: -0.1269

Data Index: 4537401 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7562. Min Act: -0.1080

Data Index: 4023779 (Open Web Text)

Max Activating Token Index: 535

Text #15

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7460. Min Act: -0.0749

Data Index: 4948730 (Open Web Text)

Max Activating Token Index: 555

Text #16

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7518. Min Act: -0.1109

Data Index: 5860013 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7400. Min Act: -0.0777

Data Index: 6021565 (Open Web Text)

Max Activating Token Index: 1023

Text #18

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7374. Min Act: -0.0927

Data Index: 675546 (Open Web Text)

Max Activating Token Index: 1023

Text #19

Max Range: 0.8467. Min Range: -0.8467

Max Act: 0.7363. Min Act: -0.0864

Data Index: 8028421 (Open Web Text)

Max Activating Token Index: 518