Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2070 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.6620. Min Act: -0.1700

Data Index: 5057530 (Open Web Text)

Max Activating Token Index: 340

Text #1

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5985. Min Act: -0.1700

Data Index: 6716330 (Open Web Text)

Max Activating Token Index: 1004

Text #2

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5865. Min Act: -0.1700

Data Index: 3099019 (Open Web Text)

Max Activating Token Index: 170

Text #3

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5833. Min Act: -0.1700

Data Index: 2880884 (Open Web Text)

Max Activating Token Index: 446

Text #4

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5544. Min Act: -0.1700

Data Index: 1060722 (Open Web Text)

Max Activating Token Index: 818

Text #5

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5707. Min Act: -0.1700

Data Index: 6815523 (Open Web Text)

Max Activating Token Index: 926

Text #6

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5501. Min Act: -0.1700

Data Index: 872200 (Open Web Text)

Max Activating Token Index: 756

Text #7

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5460. Min Act: -0.1700

Data Index: 8595 (Open Web Text)

Max Activating Token Index: 408

Text #8

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5423. Min Act: -0.1700

Data Index: 8549859 (Open Web Text)

Max Activating Token Index: 256

Text #9

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5462. Min Act: -0.1700

Data Index: 7036578 (Open Web Text)

Max Activating Token Index: 977

Text #10

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5132. Min Act: -0.1700

Data Index: 317067 (Open Web Text)

Max Activating Token Index: 555

Text #11

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5165. Min Act: -0.1700

Data Index: 3272368 (Open Web Text)

Max Activating Token Index: 335

Text #12

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5145. Min Act: -0.1700

Data Index: 8525168 (Open Web Text)

Max Activating Token Index: 494

Text #13

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4946. Min Act: -0.1700

Data Index: 32606 (Open Web Text)

Max Activating Token Index: 903

Text #14

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.5046. Min Act: -0.1700

Data Index: 787588 (Open Web Text)

Max Activating Token Index: 8

Text #15

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4805. Min Act: -0.1700

Data Index: 171013 (Open Web Text)

Max Activating Token Index: 158

Text #16

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4825. Min Act: -0.1700

Data Index: 5086417 (Open Web Text)

Max Activating Token Index: 851

Text #17

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4672. Min Act: -0.1700

Data Index: 1122587 (Open Web Text)

Max Activating Token Index: 257

Text #18

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4711. Min Act: -0.1700

Data Index: 6906266 (Open Web Text)

Max Activating Token Index: 578

Text #19

Max Range: 3.6620. Min Range: -3.6620

Max Act: 3.4762. Min Act: -0.1700

Data Index: 3320505 (Open Web Text)

Max Activating Token Index: 794