Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3353 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.4287. Min Act: -0.0000

Data Index: 636607 (The Pile)

Max Activating Token Index: 195

Text #1

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.4359. Min Act: -0.0000

Data Index: 223545 (The Pile)

Max Activating Token Index: 226

Text #2

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3403. Min Act: -0.0000

Data Index: 30077 (The Pile)

Max Activating Token Index: 983

Text #3

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3944. Min Act: -0.0000

Data Index: 409584 (The Pile)

Max Activating Token Index: 589

Text #4

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3550. Min Act: -0.0000

Data Index: 19437 (The Pile)

Max Activating Token Index: 254

Text #5

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3351. Min Act: -0.0000

Data Index: 616455 (The Pile)

Max Activating Token Index: 435

Text #6

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3549. Min Act: -0.0000

Data Index: 530744 (The Pile)

Max Activating Token Index: 754

Text #7

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3495. Min Act: -0.0000

Data Index: 1172116 (The Pile)

Max Activating Token Index: 226

Text #8

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3857. Min Act: -0.0000

Data Index: 1430583 (The Pile)

Max Activating Token Index: 419

Text #9

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3543. Min Act: -0.0000

Data Index: 944252 (The Pile)

Max Activating Token Index: 898

Text #10

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3404. Min Act: -0.0000

Data Index: 1642542 (The Pile)

Max Activating Token Index: 924

Text #11

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3607. Min Act: -0.0000

Data Index: 962933 (The Pile)

Max Activating Token Index: 754

Text #12

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3532. Min Act: -0.0000

Data Index: 28420 (The Pile)

Max Activating Token Index: 182

Text #13

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3496. Min Act: -0.0000

Data Index: 760448 (The Pile)

Max Activating Token Index: 406

Text #14

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3480. Min Act: -0.0000

Data Index: 1452201 (The Pile)

Max Activating Token Index: 405

Text #15

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3348. Min Act: -0.0000

Data Index: 108228 (The Pile)

Max Activating Token Index: 513

Text #16

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3076. Min Act: -0.0000

Data Index: 1512755 (The Pile)

Max Activating Token Index: 759

Text #17

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3610. Min Act: -0.0000

Data Index: 21535 (The Pile)

Max Activating Token Index: 113

Text #18

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3506. Min Act: -0.0000

Data Index: 107303 (The Pile)

Max Activating Token Index: 763

Text #19

Max Range: 0.4359. Min Range: -0.4359

Max Act: 0.3438. Min Act: -0.0000

Data Index: 1903839 (The Pile)

Max Activating Token Index: 120