Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 831 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.8526. Min Act: -0.1700

Data Index: 5775798 (Open Web Text)

Max Activating Token Index: 208

Text #1

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7921. Min Act: -0.1700

Data Index: 7350334 (Open Web Text)

Max Activating Token Index: 424

Text #2

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7860. Min Act: -0.1700

Data Index: 5225979 (Open Web Text)

Max Activating Token Index: 803

Text #3

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7157. Min Act: -0.1700

Data Index: 4914470 (Open Web Text)

Max Activating Token Index: 259

Text #4

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7616. Min Act: -0.1700

Data Index: 7463004 (Open Web Text)

Max Activating Token Index: 854

Text #5

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7199. Min Act: -0.1700

Data Index: 6941979 (Open Web Text)

Max Activating Token Index: 922

Text #6

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6401. Min Act: -0.1700

Data Index: 1509784 (Open Web Text)

Max Activating Token Index: 488

Text #7

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6396. Min Act: -0.1700

Data Index: 1694638 (Open Web Text)

Max Activating Token Index: 321

Text #8

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6660. Min Act: -0.1700

Data Index: 8294261 (Open Web Text)

Max Activating Token Index: 517

Text #9

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.7109. Min Act: -0.1700

Data Index: 3676844 (Open Web Text)

Max Activating Token Index: 410

Text #10

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6715. Min Act: -0.1700

Data Index: 8443221 (Open Web Text)

Max Activating Token Index: 941

Text #11

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6017. Min Act: -0.1700

Data Index: 635997 (Open Web Text)

Max Activating Token Index: 893

Text #12

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6053. Min Act: -0.1700

Data Index: 6823348 (Open Web Text)

Max Activating Token Index: 795

Text #13

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.5948. Min Act: -0.1700

Data Index: 4804714 (Open Web Text)

Max Activating Token Index: 509

Text #14

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6014. Min Act: -0.1700

Data Index: 3500029 (Open Web Text)

Max Activating Token Index: 472

Text #15

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6216. Min Act: -0.1700

Data Index: 4280078 (Open Web Text)

Max Activating Token Index: 240

Text #16

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6003. Min Act: -0.1700

Data Index: 858486 (Open Web Text)

Max Activating Token Index: 627

Text #17

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.6101. Min Act: -0.1700

Data Index: 5115312 (Open Web Text)

Max Activating Token Index: 589

Text #18

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.5489. Min Act: -0.1700

Data Index: 437383 (Open Web Text)

Max Activating Token Index: 814

Text #19

Max Range: 4.8526. Min Range: -4.8526

Max Act: 4.5955. Min Act: -0.1700

Data Index: 1804966 (Open Web Text)

Max Activating Token Index: 752