Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2299 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2420. Min Range: -5.2420

Max Act: 5.2420. Min Act: -0.1700

Data Index: 6813188 (Open Web Text)

Max Activating Token Index: 770

Text #1

Max Range: 5.2420. Min Range: -5.2420

Max Act: 5.1152. Min Act: -0.1700

Data Index: 996034 (Open Web Text)

Max Activating Token Index: 611

Text #2

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.9541. Min Act: -0.1700

Data Index: 4256937 (Open Web Text)

Max Activating Token Index: 68

Text #3

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.8895. Min Act: -0.1700

Data Index: 3140450 (Open Web Text)

Max Activating Token Index: 217

Text #4

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.8254. Min Act: -0.1700

Data Index: 3013646 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.7788. Min Act: -0.1700

Data Index: 981677 (Open Web Text)

Max Activating Token Index: 377

Text #6

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.7266. Min Act: -0.1700

Data Index: 2246047 (Open Web Text)

Max Activating Token Index: 486

Text #7

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.7197. Min Act: -0.1700

Data Index: 2232202 (Open Web Text)

Max Activating Token Index: 264

Text #8

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.7235. Min Act: -0.1700

Data Index: 7022633 (Open Web Text)

Max Activating Token Index: 107

Text #9

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6750. Min Act: -0.1700

Data Index: 124746 (Open Web Text)

Max Activating Token Index: 650

Text #10

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6510. Min Act: -0.1700

Data Index: 1046860 (Open Web Text)

Max Activating Token Index: 193

Text #11

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6613. Min Act: -0.1700

Data Index: 4988019 (Open Web Text)

Max Activating Token Index: 656

Text #12

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6423. Min Act: -0.1700

Data Index: 6830983 (Open Web Text)

Max Activating Token Index: 556

Text #13

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6922. Min Act: -0.1700

Data Index: 5638905 (Open Web Text)

Max Activating Token Index: 255

Text #14

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6126. Min Act: -0.1700

Data Index: 1685797 (Open Web Text)

Max Activating Token Index: 680

Text #15

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6110. Min Act: -0.1700

Data Index: 6671135 (Open Web Text)

Max Activating Token Index: 469

Text #16

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6043. Min Act: -0.1700

Data Index: 4221229 (Open Web Text)

Max Activating Token Index: 63

Text #17

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.6177. Min Act: -0.1700

Data Index: 1110903 (Open Web Text)

Max Activating Token Index: 682

Text #18

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.5873. Min Act: -0.1700

Data Index: 4205090 (Open Web Text)

Max Activating Token Index: 412

Text #19

Max Range: 5.2420. Min Range: -5.2420

Max Act: 4.5804. Min Act: -0.1700

Data Index: 1632347 (Open Web Text)

Max Activating Token Index: 303