Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2603 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.7772. Min Act: -0.1700

Data Index: 2212523 (Open Web Text)

Max Activating Token Index: 647

Text #1

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.7197. Min Act: -0.1700

Data Index: 2238112 (Open Web Text)

Max Activating Token Index: 78

Text #2

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.7078. Min Act: -0.1700

Data Index: 470405 (Open Web Text)

Max Activating Token Index: 622

Text #3

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.7100. Min Act: -0.1700

Data Index: 1042817 (Open Web Text)

Max Activating Token Index: 645

Text #4

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6741. Min Act: -0.1700

Data Index: 3618414 (Open Web Text)

Max Activating Token Index: 82

Text #5

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6871. Min Act: -0.1700

Data Index: 6783899 (Open Web Text)

Max Activating Token Index: 334

Text #6

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6322. Min Act: -0.1700

Data Index: 8600260 (Open Web Text)

Max Activating Token Index: 506

Text #7

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6279. Min Act: -0.1700

Data Index: 8544261 (Open Web Text)

Max Activating Token Index: 183

Text #8

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6449. Min Act: -0.1700

Data Index: 2446858 (Open Web Text)

Max Activating Token Index: 370

Text #9

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.6207. Min Act: -0.1700

Data Index: 6061130 (Open Web Text)

Max Activating Token Index: 164

Text #10

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5598. Min Act: -0.1700

Data Index: 6903722 (Open Web Text)

Max Activating Token Index: 432

Text #11

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5768. Min Act: -0.1700

Data Index: 4086966 (Open Web Text)

Max Activating Token Index: 250

Text #12

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5576. Min Act: -0.1700

Data Index: 3971328 (Open Web Text)

Max Activating Token Index: 467

Text #13

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5278. Min Act: -0.1700

Data Index: 1599950 (Open Web Text)

Max Activating Token Index: 275

Text #14

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5423. Min Act: -0.1700

Data Index: 7789380 (Open Web Text)

Max Activating Token Index: 621

Text #15

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5210. Min Act: -0.1700

Data Index: 1483115 (Open Web Text)

Max Activating Token Index: 696

Text #16

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.4897. Min Act: -0.1700

Data Index: 336365 (Open Web Text)

Max Activating Token Index: 944

Text #17

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5160. Min Act: -0.1700

Data Index: 3932476 (Open Web Text)

Max Activating Token Index: 155

Text #18

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.5041. Min Act: -0.1700

Data Index: 6835255 (Open Web Text)

Max Activating Token Index: 340

Text #19

Max Range: 3.7772. Min Range: -3.7772

Max Act: 3.4737. Min Act: -0.1700

Data Index: 7211371 (Open Web Text)

Max Activating Token Index: 511