Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3065 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0976. Min Range: -3.0976

Max Act: 3.0976. Min Act: -0.1700

Data Index: 6040302 (Open Web Text)

Max Activating Token Index: 827

Text #1

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.8293. Min Act: -0.1700

Data Index: 5473994 (Open Web Text)

Max Activating Token Index: 949

Text #2

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.8231. Min Act: -0.1700

Data Index: 5895828 (Open Web Text)

Max Activating Token Index: 129

Text #3

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.8073. Min Act: -0.1700

Data Index: 3791406 (Open Web Text)

Max Activating Token Index: 275

Text #4

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7732. Min Act: -0.1700

Data Index: 1092001 (Open Web Text)

Max Activating Token Index: 319

Text #5

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7778. Min Act: -0.1700

Data Index: 3880769 (Open Web Text)

Max Activating Token Index: 903

Text #6

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7730. Min Act: -0.1700

Data Index: 7740157 (Open Web Text)

Max Activating Token Index: 503

Text #7

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7604. Min Act: -0.1700

Data Index: 6823900 (Open Web Text)

Max Activating Token Index: 1000

Text #8

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7445. Min Act: -0.1700

Data Index: 7121553 (Open Web Text)

Max Activating Token Index: 222

Text #9

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7513. Min Act: -0.1700

Data Index: 73771 (Open Web Text)

Max Activating Token Index: 476

Text #10

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7304. Min Act: -0.1700

Data Index: 1948018 (Open Web Text)

Max Activating Token Index: 192

Text #11

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7342. Min Act: -0.1700

Data Index: 2391726 (Open Web Text)

Max Activating Token Index: 336

Text #12

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7199. Min Act: -0.1700

Data Index: 7957921 (Open Web Text)

Max Activating Token Index: 870

Text #13

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6956. Min Act: -0.1700

Data Index: 4397335 (Open Web Text)

Max Activating Token Index: 142

Text #14

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.7039. Min Act: -0.1700

Data Index: 6640685 (Open Web Text)

Max Activating Token Index: 554

Text #15

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6772. Min Act: -0.1700

Data Index: 6201312 (Open Web Text)

Max Activating Token Index: 362

Text #16

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6911. Min Act: -0.1700

Data Index: 8050676 (Open Web Text)

Max Activating Token Index: 443

Text #17

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6691. Min Act: -0.1700

Data Index: 4870337 (Open Web Text)

Max Activating Token Index: 375

Text #18

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6792. Min Act: -0.1700

Data Index: 6764983 (Open Web Text)

Max Activating Token Index: 614

Text #19

Max Range: 3.0976. Min Range: -3.0976

Max Act: 2.6497. Min Act: -0.1700

Data Index: 4093513 (Open Web Text)

Max Activating Token Index: 169