Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 22 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.7901. Min Act: -0.1700

Data Index: 259249 (C4 (Web Text))

Max Activating Token Index: 838

Text #1

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.7724. Min Act: -0.1700

Data Index: 66055 (C4 (Web Text))

Max Activating Token Index: 226

Text #2

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.7746. Min Act: -0.1700

Data Index: 565102 (C4 (Web Text))

Max Activating Token Index: 456

Text #3

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.7383. Min Act: -0.1700

Data Index: 323369 (C4 (Web Text))

Max Activating Token Index: 223

Text #4

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.6258. Min Act: -0.1700

Data Index: 130903 (C4 (Web Text))

Max Activating Token Index: 80

Text #5

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.6246. Min Act: -0.1700

Data Index: 1242137 (C4 (Web Text))

Max Activating Token Index: 317

Text #6

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.6058. Min Act: -0.1700

Data Index: 394891 (C4 (Web Text))

Max Activating Token Index: 37

Text #7

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5783. Min Act: -0.1700

Data Index: 1240744 (C4 (Web Text))

Max Activating Token Index: 720

Text #8

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5751. Min Act: -0.1700

Data Index: 1046858 (C4 (Web Text))

Max Activating Token Index: 57

Text #9

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5867. Min Act: -0.1700

Data Index: 1059663 (C4 (Web Text))

Max Activating Token Index: 922

Text #10

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5801. Min Act: -0.1700

Data Index: 1326809 (C4 (Web Text))

Max Activating Token Index: 760

Text #11

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5848. Min Act: -0.1700

Data Index: 249566 (C4 (Web Text))

Max Activating Token Index: 124

Text #12

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5510. Min Act: -0.1700

Data Index: 764127 (C4 (Web Text))

Max Activating Token Index: 80

Text #13

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5448. Min Act: -0.1700

Data Index: 1135043 (C4 (Web Text))

Max Activating Token Index: 715

Text #14

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5484. Min Act: -0.1700

Data Index: 302766 (C4 (Web Text))

Max Activating Token Index: 637

Text #15

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5444. Min Act: -0.1700

Data Index: 792757 (C4 (Web Text))

Max Activating Token Index: 419

Text #16

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5423. Min Act: -0.1700

Data Index: 686745 (C4 (Web Text))

Max Activating Token Index: 451

Text #17

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5365. Min Act: -0.1700

Data Index: 64501 (C4 (Web Text))

Max Activating Token Index: 750

Text #18

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5359. Min Act: -0.1700

Data Index: 338906 (C4 (Web Text))

Max Activating Token Index: 759

Text #19

Max Range: 3.7901. Min Range: -3.7901

Max Act: 3.5286. Min Act: -0.1700

Data Index: 47966 (C4 (Web Text))

Max Activating Token Index: 255