Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 747 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7791. Min Act: -0.1700

Data Index: 6439130 (Open Web Text)

Max Activating Token Index: 675

Text #1

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.8098. Min Act: -0.1700

Data Index: 6808089 (Open Web Text)

Max Activating Token Index: 402

Text #2

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7986. Min Act: -0.1700

Data Index: 4466841 (Open Web Text)

Max Activating Token Index: 436

Text #3

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7768. Min Act: -0.1700

Data Index: 6616591 (Open Web Text)

Max Activating Token Index: 384

Text #4

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7813. Min Act: -0.1700

Data Index: 6883695 (Open Web Text)

Max Activating Token Index: 263

Text #5

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7712. Min Act: -0.1700

Data Index: 3147053 (Open Web Text)

Max Activating Token Index: 543

Text #6

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7483. Min Act: -0.1700

Data Index: 8562746 (Open Web Text)

Max Activating Token Index: 147

Text #7

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7796. Min Act: -0.1700

Data Index: 4847800 (Open Web Text)

Max Activating Token Index: 623

Text #8

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7507. Min Act: -0.1700

Data Index: 4790789 (Open Web Text)

Max Activating Token Index: 172

Text #9

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7280. Min Act: -0.1700

Data Index: 2287093 (Open Web Text)

Max Activating Token Index: 470

Text #10

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7682. Min Act: -0.1700

Data Index: 7641013 (Open Web Text)

Max Activating Token Index: 100

Text #11

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7462. Min Act: -0.1700

Data Index: 955456 (Open Web Text)

Max Activating Token Index: 309

Text #12

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7498. Min Act: -0.1700

Data Index: 6911156 (Open Web Text)

Max Activating Token Index: 520

Text #13

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7486. Min Act: -0.1700

Data Index: 6107237 (Open Web Text)

Max Activating Token Index: 185

Text #14

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7678. Min Act: -0.1700

Data Index: 3023787 (Open Web Text)

Max Activating Token Index: 41

Text #15

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.6787. Min Act: -0.1700

Data Index: 7892630 (Open Web Text)

Max Activating Token Index: 735

Text #16

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7307. Min Act: -0.1700

Data Index: 8466538 (Open Web Text)

Max Activating Token Index: 936

Text #17

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7290. Min Act: -0.1700

Data Index: 930652 (Open Web Text)

Max Activating Token Index: 873

Text #18

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7009. Min Act: -0.1700

Data Index: 1332206 (Open Web Text)

Max Activating Token Index: 645

Text #19

Max Range: 3.8098. Min Range: -3.8098

Max Act: 3.7127. Min Act: -0.1700

Data Index: 2984841 (Open Web Text)

Max Activating Token Index: 256