Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1206 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1145. Min Range: -4.1145

Max Act: 4.1145. Min Act: -0.1700

Data Index: 3622670 (Open Web Text)

Max Activating Token Index: 648

Text #1

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.9520. Min Act: -0.1700

Data Index: 898673 (Open Web Text)

Max Activating Token Index: 1019

Text #2

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.8832. Min Act: -0.1700

Data Index: 1829035 (Open Web Text)

Max Activating Token Index: 979

Text #3

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.8542. Min Act: -0.1700

Data Index: 796532 (Open Web Text)

Max Activating Token Index: 820

Text #4

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.8400. Min Act: -0.1700

Data Index: 3845995 (Open Web Text)

Max Activating Token Index: 539

Text #5

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.8287. Min Act: -0.1700

Data Index: 1751457 (Open Web Text)

Max Activating Token Index: 832

Text #6

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.8299. Min Act: -0.1700

Data Index: 4541000 (Open Web Text)

Max Activating Token Index: 749

Text #7

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7934. Min Act: -0.1700

Data Index: 4179788 (Open Web Text)

Max Activating Token Index: 684

Text #8

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7592. Min Act: -0.1700

Data Index: 7434504 (Open Web Text)

Max Activating Token Index: 1003

Text #9

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7865. Min Act: -0.1700

Data Index: 6265723 (Open Web Text)

Max Activating Token Index: 981

Text #10

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7555. Min Act: -0.1700

Data Index: 887710 (Open Web Text)

Max Activating Token Index: 955

Text #11

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7824. Min Act: -0.1700

Data Index: 193341 (Open Web Text)

Max Activating Token Index: 788

Text #12

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7857. Min Act: -0.1700

Data Index: 5173179 (Open Web Text)

Max Activating Token Index: 976

Text #13

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7064. Min Act: -0.1700

Data Index: 6070663 (Open Web Text)

Max Activating Token Index: 983

Text #14

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7636. Min Act: -0.1700

Data Index: 7984735 (Open Web Text)

Max Activating Token Index: 610

Text #15

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7122. Min Act: -0.1700

Data Index: 2636748 (Open Web Text)

Max Activating Token Index: 772

Text #16

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.6980. Min Act: -0.1700

Data Index: 8248798 (Open Web Text)

Max Activating Token Index: 976

Text #17

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7064. Min Act: -0.1700

Data Index: 2548010 (Open Web Text)

Max Activating Token Index: 977

Text #18

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.7049. Min Act: -0.1700

Data Index: 4765752 (Open Web Text)

Max Activating Token Index: 802

Text #19

Max Range: 4.1145. Min Range: -4.1145

Max Act: 3.6877. Min Act: -0.1700

Data Index: 5175362 (Open Web Text)

Max Activating Token Index: 821