Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3013 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.5006. Min Act: -0.1700

Data Index: 1284721 (Open Web Text)

Max Activating Token Index: 295

Text #1

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.4182. Min Act: -0.1700

Data Index: 3924957 (Open Web Text)

Max Activating Token Index: 8

Text #2

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3594. Min Act: -0.1700

Data Index: 4545641 (Open Web Text)

Max Activating Token Index: 842

Text #3

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3286. Min Act: -0.1700

Data Index: 1241609 (Open Web Text)

Max Activating Token Index: 195

Text #4

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3321. Min Act: -0.1700

Data Index: 5020315 (Open Web Text)

Max Activating Token Index: 98

Text #5

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3363. Min Act: -0.1700

Data Index: 777998 (Open Web Text)

Max Activating Token Index: 306

Text #6

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2946. Min Act: -0.1700

Data Index: 2982976 (Open Web Text)

Max Activating Token Index: 125

Text #7

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2949. Min Act: -0.1700

Data Index: 5777498 (Open Web Text)

Max Activating Token Index: 522

Text #8

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3335. Min Act: -0.1700

Data Index: 3068250 (Open Web Text)

Max Activating Token Index: 873

Text #9

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3247. Min Act: -0.1700

Data Index: 7734403 (Open Web Text)

Max Activating Token Index: 153

Text #10

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2777. Min Act: -0.1700

Data Index: 6181331 (Open Web Text)

Max Activating Token Index: 751

Text #11

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2530. Min Act: -0.1700

Data Index: 2450204 (Open Web Text)

Max Activating Token Index: 698

Text #12

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2915. Min Act: -0.1700

Data Index: 8753000 (Open Web Text)

Max Activating Token Index: 297

Text #13

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2675. Min Act: -0.1700

Data Index: 8525384 (Open Web Text)

Max Activating Token Index: 487

Text #14

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2610. Min Act: -0.1700

Data Index: 7779433 (Open Web Text)

Max Activating Token Index: 753

Text #15

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2866. Min Act: -0.1700

Data Index: 6412608 (Open Web Text)

Max Activating Token Index: 561

Text #16

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2809. Min Act: -0.1700

Data Index: 6118965 (Open Web Text)

Max Activating Token Index: 183

Text #17

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2769. Min Act: -0.1700

Data Index: 1941945 (Open Web Text)

Max Activating Token Index: 17

Text #18

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.3067. Min Act: -0.1700

Data Index: 2564730 (Open Web Text)

Max Activating Token Index: 793

Text #19

Max Range: 3.5006. Min Range: -3.5006

Max Act: 3.2690. Min Act: -0.1700

Data Index: 801236 (Open Web Text)

Max Activating Token Index: 666