Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1908 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.8247. Min Act: -0.1700

Data Index: 5446122 (Open Web Text)

Max Activating Token Index: 1023

Text #1

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.8030. Min Act: -0.1700

Data Index: 6500789 (Open Web Text)

Max Activating Token Index: 216

Text #2

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7834. Min Act: -0.1700

Data Index: 8461086 (Open Web Text)

Max Activating Token Index: 1023

Text #3

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7763. Min Act: -0.1698

Data Index: 3285219 (Open Web Text)

Max Activating Token Index: 499

Text #4

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7742. Min Act: -0.1700

Data Index: 2041709 (Open Web Text)

Max Activating Token Index: 2

Text #5

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7742. Min Act: -0.1700

Data Index: 7659897 (Open Web Text)

Max Activating Token Index: 2

Text #6

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7755. Min Act: -0.1700

Data Index: 3629496 (Open Web Text)

Max Activating Token Index: 437

Text #7

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7783. Min Act: -0.1699

Data Index: 4460739 (Open Web Text)

Max Activating Token Index: 2

Text #8

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7622. Min Act: -0.1699

Data Index: 2118305 (Open Web Text)

Max Activating Token Index: 485

Text #9

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7576. Min Act: -0.1700

Data Index: 3485228 (Open Web Text)

Max Activating Token Index: 165

Text #10

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7383. Min Act: -0.1700

Data Index: 2316640 (Open Web Text)

Max Activating Token Index: 581

Text #11

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7510. Min Act: -0.1700

Data Index: 1906162 (Open Web Text)

Max Activating Token Index: 68

Text #12

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7474. Min Act: -0.1700

Data Index: 347384 (Open Web Text)

Max Activating Token Index: 752

Text #13

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7535. Min Act: -0.1700

Data Index: 2846750 (Open Web Text)

Max Activating Token Index: 1023

Text #14

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7384. Min Act: -0.1700

Data Index: 6765901 (Open Web Text)

Max Activating Token Index: 1023

Text #15

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7293. Min Act: -0.1700

Data Index: 3352107 (Open Web Text)

Max Activating Token Index: 328

Text #16

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7342. Min Act: -0.1698

Data Index: 5805349 (Open Web Text)

Max Activating Token Index: 1023

Text #17

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7368. Min Act: -0.1700

Data Index: 5127287 (Open Web Text)

Max Activating Token Index: 536

Text #18

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7280. Min Act: -0.1700

Data Index: 8272468 (Open Web Text)

Max Activating Token Index: 120

Text #19

Max Range: 1.8247. Min Range: -1.8247

Max Act: 1.7337. Min Act: -0.1700

Data Index: 8432559 (Open Web Text)

Max Activating Token Index: 332