Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 887 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.8178. Min Act: -0.1700

Data Index: 5451162 (Open Web Text)

Max Activating Token Index: 32

Text #1

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.8161. Min Act: -0.1700

Data Index: 7769713 (Open Web Text)

Max Activating Token Index: 46

Text #2

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7772. Min Act: -0.1700

Data Index: 4749264 (Open Web Text)

Max Activating Token Index: 52

Text #3

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7809. Min Act: -0.1700

Data Index: 8179995 (Open Web Text)

Max Activating Token Index: 779

Text #4

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7470. Min Act: -0.1700

Data Index: 2249947 (Open Web Text)

Max Activating Token Index: 114

Text #5

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7607. Min Act: -0.1700

Data Index: 5139773 (Open Web Text)

Max Activating Token Index: 194

Text #6

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7425. Min Act: -0.1700

Data Index: 2187906 (Open Web Text)

Max Activating Token Index: 601

Text #7

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7077. Min Act: -0.1700

Data Index: 8575899 (Open Web Text)

Max Activating Token Index: 100

Text #8

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.7131. Min Act: -0.1700

Data Index: 8378951 (Open Web Text)

Max Activating Token Index: 539

Text #9

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6784. Min Act: -0.1700

Data Index: 3751861 (Open Web Text)

Max Activating Token Index: 53

Text #10

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6918. Min Act: -0.1700

Data Index: 4048400 (Open Web Text)

Max Activating Token Index: 450

Text #11

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6907. Min Act: -0.1700

Data Index: 2249949 (Open Web Text)

Max Activating Token Index: 397

Text #12

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6917. Min Act: -0.1700

Data Index: 7236921 (Open Web Text)

Max Activating Token Index: 468

Text #13

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6808. Min Act: -0.1700

Data Index: 4991785 (Open Web Text)

Max Activating Token Index: 691

Text #14

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6898. Min Act: -0.1700

Data Index: 8652955 (Open Web Text)

Max Activating Token Index: 139

Text #15

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6727. Min Act: -0.1700

Data Index: 1902812 (Open Web Text)

Max Activating Token Index: 337

Text #16

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6362. Min Act: -0.1700

Data Index: 1398028 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6631. Min Act: -0.1700

Data Index: 1041534 (Open Web Text)

Max Activating Token Index: 592

Text #18

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6669. Min Act: -0.1700

Data Index: 1038643 (Open Web Text)

Max Activating Token Index: 17

Text #19

Max Range: 4.8178. Min Range: -4.8178

Max Act: 4.6640. Min Act: -0.1700

Data Index: 3658718 (Open Web Text)

Max Activating Token Index: 841