Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2143 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.5055. Min Act: -0.1700

Data Index: 4464854 (Open Web Text)

Max Activating Token Index: 537

Text #1

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.5044. Min Act: -0.1700

Data Index: 2857362 (Open Web Text)

Max Activating Token Index: 215

Text #2

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2511. Min Act: -0.1700

Data Index: 5247580 (Open Web Text)

Max Activating Token Index: 456

Text #3

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2721. Min Act: -0.1700

Data Index: 2704503 (Open Web Text)

Max Activating Token Index: 886

Text #4

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1894. Min Act: -0.1700

Data Index: 846344 (Open Web Text)

Max Activating Token Index: 547

Text #5

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2470. Min Act: -0.1700

Data Index: 3382225 (Open Web Text)

Max Activating Token Index: 819

Text #6

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1895. Min Act: -0.1700

Data Index: 1166836 (Open Web Text)

Max Activating Token Index: 569

Text #7

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2537. Min Act: -0.1700

Data Index: 3520227 (Open Web Text)

Max Activating Token Index: 716

Text #8

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2376. Min Act: -0.1700

Data Index: 3731835 (Open Web Text)

Max Activating Token Index: 890

Text #9

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2273. Min Act: -0.1700

Data Index: 6858491 (Open Web Text)

Max Activating Token Index: 415

Text #10

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2422. Min Act: -0.1700

Data Index: 410812 (Open Web Text)

Max Activating Token Index: 499

Text #11

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2068. Min Act: -0.1700

Data Index: 2027217 (Open Web Text)

Max Activating Token Index: 163

Text #12

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2376. Min Act: -0.1700

Data Index: 3676374 (Open Web Text)

Max Activating Token Index: 486

Text #13

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2204. Min Act: -0.1700

Data Index: 2434392 (Open Web Text)

Max Activating Token Index: 789

Text #14

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.2297. Min Act: -0.1700

Data Index: 2046128 (Open Web Text)

Max Activating Token Index: 741

Text #15

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1895. Min Act: -0.1700

Data Index: 386745 (Open Web Text)

Max Activating Token Index: 847

Text #16

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1544. Min Act: -0.1700

Data Index: 2709281 (Open Web Text)

Max Activating Token Index: 856

Text #17

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1827. Min Act: -0.1700

Data Index: 2326556 (Open Web Text)

Max Activating Token Index: 413

Text #18

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1823. Min Act: -0.1700

Data Index: 8753838 (Open Web Text)

Max Activating Token Index: 1021

Text #19

Max Range: 4.5055. Min Range: -4.5055

Max Act: 4.1211. Min Act: -0.1700

Data Index: 14643 (Open Web Text)

Max Activating Token Index: 622