Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2331 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.7525. Min Act: -0.1700

Data Index: 2859025 (Open Web Text)

Max Activating Token Index: 540

Text #1

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.4086. Min Act: -0.1700

Data Index: 1942280 (Open Web Text)

Max Activating Token Index: 981

Text #2

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3758. Min Act: -0.1700

Data Index: 8711527 (Open Web Text)

Max Activating Token Index: 205

Text #3

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.4052. Min Act: -0.1700

Data Index: 1789614 (Open Web Text)

Max Activating Token Index: 967

Text #4

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3789. Min Act: -0.1700

Data Index: 2210335 (Open Web Text)

Max Activating Token Index: 941

Text #5

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3583. Min Act: -0.1700

Data Index: 2690393 (Open Web Text)

Max Activating Token Index: 29

Text #6

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3182. Min Act: -0.1700

Data Index: 5900316 (Open Web Text)

Max Activating Token Index: 798

Text #7

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3292. Min Act: -0.1700

Data Index: 1195820 (Open Web Text)

Max Activating Token Index: 146

Text #8

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3313. Min Act: -0.1700

Data Index: 1273451 (Open Web Text)

Max Activating Token Index: 637

Text #9

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3146. Min Act: -0.1700

Data Index: 508554 (Open Web Text)

Max Activating Token Index: 532

Text #10

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3248. Min Act: -0.1700

Data Index: 1815042 (Open Web Text)

Max Activating Token Index: 321

Text #11

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2922. Min Act: -0.1700

Data Index: 702334 (Open Web Text)

Max Activating Token Index: 96

Text #12

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.3012. Min Act: -0.1700

Data Index: 7690573 (Open Web Text)

Max Activating Token Index: 42

Text #13

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2832. Min Act: -0.1700

Data Index: 7069057 (Open Web Text)

Max Activating Token Index: 246

Text #14

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2782. Min Act: -0.1700

Data Index: 7927079 (Open Web Text)

Max Activating Token Index: 721

Text #15

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2986. Min Act: -0.1700

Data Index: 4532759 (Open Web Text)

Max Activating Token Index: 17

Text #16

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2987. Min Act: -0.1700

Data Index: 111023 (Open Web Text)

Max Activating Token Index: 844

Text #17

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2783. Min Act: -0.1700

Data Index: 4739224 (Open Web Text)

Max Activating Token Index: 838

Text #18

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2839. Min Act: -0.1700

Data Index: 1858677 (Open Web Text)

Max Activating Token Index: 25

Text #19

Max Range: 3.7525. Min Range: -3.7525

Max Act: 3.2920. Min Act: -0.1700

Data Index: 5283196 (Open Web Text)

Max Activating Token Index: 876