Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1411 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3574. Min Act: -0.1700

Data Index: 7035875 (Open Web Text)

Max Activating Token Index: 993

Text #1

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3573. Min Act: -0.1700

Data Index: 7490393 (Open Web Text)

Max Activating Token Index: 987

Text #2

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3650. Min Act: -0.1700

Data Index: 5665871 (Open Web Text)

Max Activating Token Index: 1008

Text #3

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3520. Min Act: -0.1700

Data Index: 851987 (Open Web Text)

Max Activating Token Index: 983

Text #4

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3697. Min Act: -0.1700

Data Index: 3028412 (Open Web Text)

Max Activating Token Index: 1022

Text #5

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3771. Min Act: -0.1699

Data Index: 3902554 (Open Web Text)

Max Activating Token Index: 967

Text #6

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3219. Min Act: -0.1700

Data Index: 5051379 (Open Web Text)

Max Activating Token Index: 1016

Text #7

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3436. Min Act: -0.1700

Data Index: 4907768 (Open Web Text)

Max Activating Token Index: 1013

Text #8

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3201. Min Act: -0.1700

Data Index: 4324984 (Open Web Text)

Max Activating Token Index: 1020

Text #9

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3209. Min Act: -0.1700

Data Index: 1028056 (Open Web Text)

Max Activating Token Index: 1015

Text #10

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3261. Min Act: -0.1700

Data Index: 4122136 (Open Web Text)

Max Activating Token Index: 1012

Text #11

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3172. Min Act: -0.1700

Data Index: 3855190 (Open Web Text)

Max Activating Token Index: 1020

Text #12

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3382. Min Act: -0.1700

Data Index: 132417 (Open Web Text)

Max Activating Token Index: 1000

Text #13

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3373. Min Act: -0.1700

Data Index: 3739495 (Open Web Text)

Max Activating Token Index: 1018

Text #14

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3458. Min Act: -0.1700

Data Index: 3278715 (Open Web Text)

Max Activating Token Index: 1019

Text #15

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3267. Min Act: -0.1700

Data Index: 3233359 (Open Web Text)

Max Activating Token Index: 1012

Text #16

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3131. Min Act: -0.1700

Data Index: 2913575 (Open Web Text)

Max Activating Token Index: 1014

Text #17

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3130. Min Act: -0.1700

Data Index: 888956 (Open Web Text)

Max Activating Token Index: 1015

Text #18

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3239. Min Act: -0.1700

Data Index: 2819337 (Open Web Text)

Max Activating Token Index: 1006

Text #19

Max Range: 3.3771. Min Range: -3.3771

Max Act: 3.3296. Min Act: -0.1700

Data Index: 2795456 (Open Web Text)

Max Activating Token Index: 994