Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1790 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7929. Min Act: -0.1700

Data Index: 3151677 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7776. Min Act: -0.1700

Data Index: 2243530 (Open Web Text)

Max Activating Token Index: 6

Text #2

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7594. Min Act: -0.1700

Data Index: 1803426 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7268. Min Act: -0.1700

Data Index: 2714882 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7142. Min Act: -0.1700

Data Index: 3516827 (Open Web Text)

Max Activating Token Index: 5

Text #5

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6907. Min Act: -0.1700

Data Index: 5904233 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.7110. Min Act: -0.1700

Data Index: 5695933 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6522. Min Act: -0.1700

Data Index: 7157118 (Open Web Text)

Max Activating Token Index: 7

Text #8

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6653. Min Act: -0.1700

Data Index: 7550844 (Open Web Text)

Max Activating Token Index: 3

Text #9

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6781. Min Act: -0.1700

Data Index: 1280701 (Open Web Text)

Max Activating Token Index: 12

Text #10

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6573. Min Act: -0.1700

Data Index: 5381914 (Open Web Text)

Max Activating Token Index: 12

Text #11

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6466. Min Act: -0.1700

Data Index: 6974741 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6439. Min Act: -0.1700

Data Index: 974714 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6250. Min Act: -0.1700

Data Index: 7539566 (Open Web Text)

Max Activating Token Index: 19

Text #14

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6465. Min Act: -0.1700

Data Index: 7645473 (Open Web Text)

Max Activating Token Index: 3

Text #15

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6587. Min Act: -0.1700

Data Index: 8761540 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6332. Min Act: -0.1700

Data Index: 2958462 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.5965. Min Act: -0.1700

Data Index: 3257456 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6027. Min Act: -0.1700

Data Index: 2916815 (Open Web Text)

Max Activating Token Index: 7

Text #19

Max Range: 3.7929. Min Range: -3.7929

Max Act: 3.6213. Min Act: -0.1700

Data Index: 1992821 (Open Web Text)

Max Activating Token Index: 47