Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2358 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.7774. Min Act: -0.1700

Data Index: 5161073 (Open Web Text)

Max Activating Token Index: 385

Text #1

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.6922. Min Act: -0.1700

Data Index: 8610425 (Open Web Text)

Max Activating Token Index: 980

Text #2

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.5371. Min Act: -0.1700

Data Index: 717713 (Open Web Text)

Max Activating Token Index: 982

Text #3

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.5061. Min Act: -0.1700

Data Index: 8622407 (Open Web Text)

Max Activating Token Index: 441

Text #4

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.4800. Min Act: -0.1700

Data Index: 6187340 (Open Web Text)

Max Activating Token Index: 980

Text #5

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.4770. Min Act: -0.1700

Data Index: 4648097 (Open Web Text)

Max Activating Token Index: 995

Text #6

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.4304. Min Act: -0.1700

Data Index: 4543222 (Open Web Text)

Max Activating Token Index: 397

Text #7

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3969. Min Act: -0.1700

Data Index: 7187613 (Open Web Text)

Max Activating Token Index: 25

Text #8

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3949. Min Act: -0.1700

Data Index: 5304010 (Open Web Text)

Max Activating Token Index: 397

Text #9

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3492. Min Act: -0.1700

Data Index: 5854351 (Open Web Text)

Max Activating Token Index: 32

Text #10

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3469. Min Act: -0.1700

Data Index: 7499130 (Open Web Text)

Max Activating Token Index: 416

Text #11

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3175. Min Act: -0.1700

Data Index: 3308273 (Open Web Text)

Max Activating Token Index: 24

Text #12

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3192. Min Act: -0.1700

Data Index: 2737100 (Open Web Text)

Max Activating Token Index: 30

Text #13

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3057. Min Act: -0.1700

Data Index: 3714104 (Open Web Text)

Max Activating Token Index: 24

Text #14

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3023. Min Act: -0.1700

Data Index: 6361220 (Open Web Text)

Max Activating Token Index: 26

Text #15

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3048. Min Act: -0.1700

Data Index: 8057660 (Open Web Text)

Max Activating Token Index: 29

Text #16

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.2946. Min Act: -0.1700

Data Index: 6038461 (Open Web Text)

Max Activating Token Index: 24

Text #17

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.3033. Min Act: -0.1700

Data Index: 2783714 (Open Web Text)

Max Activating Token Index: 949

Text #18

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.2985. Min Act: -0.1700

Data Index: 24824 (Open Web Text)

Max Activating Token Index: 20

Text #19

Max Range: 0.7774. Min Range: -0.7774

Max Act: 0.2975. Min Act: -0.1700

Data Index: 3563182 (Open Web Text)

Max Activating Token Index: 22