Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2810 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0390. Min Range: -3.0390

Max Act: 3.0390. Min Act: -0.1700

Data Index: 7238786 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.9552. Min Act: -0.1698

Data Index: 859749 (Open Web Text)

Max Activating Token Index: 841

Text #2

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.9076. Min Act: -0.1700

Data Index: 507773 (Open Web Text)

Max Activating Token Index: 5

Text #3

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.9042. Min Act: -0.1700

Data Index: 7485029 (Open Web Text)

Max Activating Token Index: 871

Text #4

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.8778. Min Act: -0.1700

Data Index: 8240365 (Open Web Text)

Max Activating Token Index: 114

Text #5

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.8743. Min Act: -0.1700

Data Index: 2080843 (Open Web Text)

Max Activating Token Index: 800

Text #6

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.8580. Min Act: -0.1700

Data Index: 3623843 (Open Web Text)

Max Activating Token Index: 658

Text #7

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.8330. Min Act: -0.1700

Data Index: 2555404 (Open Web Text)

Max Activating Token Index: 419

Text #8

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7936. Min Act: -0.1700

Data Index: 7488986 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7773. Min Act: -0.1700

Data Index: 8407421 (Open Web Text)

Max Activating Token Index: 373

Text #10

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7931. Min Act: -0.1700

Data Index: 1738555 (Open Web Text)

Max Activating Token Index: 452

Text #11

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7849. Min Act: -0.1700

Data Index: 3105822 (Open Web Text)

Max Activating Token Index: 13

Text #12

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7584. Min Act: -0.1700

Data Index: 1195465 (Open Web Text)

Max Activating Token Index: 624

Text #13

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7289. Min Act: -0.1700

Data Index: 1088807 (Open Web Text)

Max Activating Token Index: 282

Text #14

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7373. Min Act: -0.1700

Data Index: 6633323 (Open Web Text)

Max Activating Token Index: 187

Text #15

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7294. Min Act: -0.1700

Data Index: 8391729 (Open Web Text)

Max Activating Token Index: 340

Text #16

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7158. Min Act: -0.1700

Data Index: 1514936 (Open Web Text)

Max Activating Token Index: 546

Text #17

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7008. Min Act: -0.1700

Data Index: 3346800 (Open Web Text)

Max Activating Token Index: 211

Text #18

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.7052. Min Act: -0.1700

Data Index: 7175586 (Open Web Text)

Max Activating Token Index: 820

Text #19

Max Range: 3.0390. Min Range: -3.0390

Max Act: 2.6698. Min Act: -0.1700

Data Index: 3970885 (Open Web Text)

Max Activating Token Index: 4