Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1140 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.5803. Min Act: -0.1700

Data Index: 3815168 (Open Web Text)

Max Activating Token Index: 146

Text #1

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.5532. Min Act: -0.1700

Data Index: 7539437 (Open Web Text)

Max Activating Token Index: 21

Text #2

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.5499. Min Act: -0.1700

Data Index: 6467405 (Open Web Text)

Max Activating Token Index: 452

Text #3

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.5286. Min Act: -0.1700

Data Index: 440234 (Open Web Text)

Max Activating Token Index: 1003

Text #4

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4868. Min Act: -0.1700

Data Index: 7949550 (Open Web Text)

Max Activating Token Index: 407

Text #5

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4733. Min Act: -0.1700

Data Index: 2562180 (Open Web Text)

Max Activating Token Index: 454

Text #6

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4839. Min Act: -0.1700

Data Index: 3574303 (Open Web Text)

Max Activating Token Index: 397

Text #7

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4909. Min Act: -0.1700

Data Index: 1324909 (Open Web Text)

Max Activating Token Index: 705

Text #8

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4894. Min Act: -0.1700

Data Index: 8220391 (Open Web Text)

Max Activating Token Index: 261

Text #9

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4586. Min Act: -0.1700

Data Index: 1500270 (Open Web Text)

Max Activating Token Index: 496

Text #10

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4447. Min Act: -0.1700

Data Index: 8073530 (Open Web Text)

Max Activating Token Index: 651

Text #11

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4604. Min Act: -0.1700

Data Index: 3645350 (Open Web Text)

Max Activating Token Index: 829

Text #12

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4559. Min Act: -0.1700

Data Index: 279396 (Open Web Text)

Max Activating Token Index: 734

Text #13

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4404. Min Act: -0.1700

Data Index: 717758 (Open Web Text)

Max Activating Token Index: 877

Text #14

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4516. Min Act: -0.1700

Data Index: 8811778 (Open Web Text)

Max Activating Token Index: 107

Text #15

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4518. Min Act: -0.1700

Data Index: 5851864 (Open Web Text)

Max Activating Token Index: 26

Text #16

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4485. Min Act: -0.1700

Data Index: 5348754 (Open Web Text)

Max Activating Token Index: 162

Text #17

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4374. Min Act: -0.1700

Data Index: 8151574 (Open Web Text)

Max Activating Token Index: 867

Text #18

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4301. Min Act: -0.1700

Data Index: 3078076 (Open Web Text)

Max Activating Token Index: 297

Text #19

Max Range: 3.5803. Min Range: -3.5803

Max Act: 3.4294. Min Act: -0.1700

Data Index: 2271261 (Open Web Text)

Max Activating Token Index: 488