Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1550 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0750. Min Range: -4.0750

Max Act: 4.0750. Min Act: -0.1700

Data Index: 4517381 (Open Web Text)

Max Activating Token Index: 890

Text #1

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9824. Min Act: -0.1700

Data Index: 1371602 (Open Web Text)

Max Activating Token Index: 310

Text #2

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9985. Min Act: -0.1700

Data Index: 2227686 (Open Web Text)

Max Activating Token Index: 835

Text #3

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9773. Min Act: -0.1700

Data Index: 162486 (Open Web Text)

Max Activating Token Index: 89

Text #4

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9434. Min Act: -0.1700

Data Index: 8426173 (Open Web Text)

Max Activating Token Index: 291

Text #5

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9020. Min Act: -0.1700

Data Index: 1462399 (Open Web Text)

Max Activating Token Index: 764

Text #6

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9006. Min Act: -0.1700

Data Index: 131019 (Open Web Text)

Max Activating Token Index: 155

Text #7

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.9042. Min Act: -0.1700

Data Index: 1239035 (Open Web Text)

Max Activating Token Index: 96

Text #8

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8742. Min Act: -0.1700

Data Index: 7478024 (Open Web Text)

Max Activating Token Index: 293

Text #9

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8668. Min Act: -0.1700

Data Index: 873956 (Open Web Text)

Max Activating Token Index: 517

Text #10

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8392. Min Act: -0.1700

Data Index: 4020872 (Open Web Text)

Max Activating Token Index: 265

Text #11

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8201. Min Act: -0.1700

Data Index: 8547192 (Open Web Text)

Max Activating Token Index: 213

Text #12

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8263. Min Act: -0.1700

Data Index: 3544299 (Open Web Text)

Max Activating Token Index: 217

Text #13

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8052. Min Act: -0.1700

Data Index: 5872910 (Open Web Text)

Max Activating Token Index: 194

Text #14

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.7998. Min Act: -0.1700

Data Index: 7882064 (Open Web Text)

Max Activating Token Index: 220

Text #15

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8176. Min Act: -0.1700

Data Index: 4633722 (Open Web Text)

Max Activating Token Index: 427

Text #16

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8095. Min Act: -0.1700

Data Index: 4246779 (Open Web Text)

Max Activating Token Index: 434

Text #17

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.7951. Min Act: -0.1700

Data Index: 690959 (Open Web Text)

Max Activating Token Index: 638

Text #18

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8036. Min Act: -0.1700

Data Index: 2043032 (Open Web Text)

Max Activating Token Index: 798

Text #19

Max Range: 4.0750. Min Range: -4.0750

Max Act: 3.8256. Min Act: -0.1700

Data Index: 3465368 (Open Web Text)

Max Activating Token Index: 488