Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2019 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.6910. Min Act: -0.1700

Data Index: 2799643 (Open Web Text)

Max Activating Token Index: 842

Text #1

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.6133. Min Act: -0.1700

Data Index: 8226343 (Open Web Text)

Max Activating Token Index: 529

Text #2

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.6101. Min Act: -0.1700

Data Index: 808032 (Open Web Text)

Max Activating Token Index: 468

Text #3

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5976. Min Act: -0.1700

Data Index: 4762320 (Open Web Text)

Max Activating Token Index: 415

Text #4

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.6320. Min Act: -0.1700

Data Index: 5374646 (Open Web Text)

Max Activating Token Index: 412

Text #5

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5911. Min Act: -0.1700

Data Index: 5650647 (Open Web Text)

Max Activating Token Index: 456

Text #6

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5674. Min Act: -0.1700

Data Index: 6461943 (Open Web Text)

Max Activating Token Index: 492

Text #7

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5687. Min Act: -0.1700

Data Index: 2564928 (Open Web Text)

Max Activating Token Index: 74

Text #8

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5431. Min Act: -0.1700

Data Index: 2727927 (Open Web Text)

Max Activating Token Index: 720

Text #9

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5559. Min Act: -0.1700

Data Index: 3764043 (Open Web Text)

Max Activating Token Index: 300

Text #10

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5322. Min Act: -0.1700

Data Index: 7240617 (Open Web Text)

Max Activating Token Index: 905

Text #11

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5196. Min Act: -0.1700

Data Index: 4787866 (Open Web Text)

Max Activating Token Index: 598

Text #12

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5362. Min Act: -0.1700

Data Index: 7660070 (Open Web Text)

Max Activating Token Index: 1014

Text #13

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5489. Min Act: -0.1700

Data Index: 5249290 (Open Web Text)

Max Activating Token Index: 548

Text #14

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5596. Min Act: -0.1700

Data Index: 794091 (Open Web Text)

Max Activating Token Index: 535

Text #15

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5415. Min Act: -0.1700

Data Index: 3494114 (Open Web Text)

Max Activating Token Index: 379

Text #16

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5518. Min Act: -0.1700

Data Index: 3909742 (Open Web Text)

Max Activating Token Index: 796

Text #17

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5503. Min Act: -0.1700

Data Index: 3261533 (Open Web Text)

Max Activating Token Index: 202

Text #18

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5227. Min Act: -0.1700

Data Index: 5875358 (Open Web Text)

Max Activating Token Index: 80

Text #19

Max Range: 3.6910. Min Range: -3.6910

Max Act: 3.5207. Min Act: -0.1700

Data Index: 3894509 (Open Web Text)

Max Activating Token Index: 165