Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3502 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0673. Min Act: 0.0000

Data Index: 66100 (The Pile)

Max Activating Token Index: 56

Text #1

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0662. Min Act: 0.0001

Data Index: 149706 (The Pile)

Max Activating Token Index: 553

Text #2

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0659. Min Act: -0.0000

Data Index: 921653 (The Pile)

Max Activating Token Index: 22

Text #3

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0572. Min Act: 0.0000

Data Index: 1747425 (The Pile)

Max Activating Token Index: 19

Text #4

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0549. Min Act: 0.0000

Data Index: 1416918 (The Pile)

Max Activating Token Index: 53

Text #5

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0526. Min Act: 0.0000

Data Index: 1019009 (The Pile)

Max Activating Token Index: 27

Text #6

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0515. Min Act: 0.0000

Data Index: 586097 (The Pile)

Max Activating Token Index: 25

Text #7

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0511. Min Act: 0.0000

Data Index: 478850 (The Pile)

Max Activating Token Index: 100

Text #8

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0506. Min Act: 0.0001

Data Index: 1232607 (The Pile)

Max Activating Token Index: 113

Text #9

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0507. Min Act: 0.0000

Data Index: 402094 (The Pile)

Max Activating Token Index: 24

Text #10

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0488. Min Act: 0.0000

Data Index: 999053 (The Pile)

Max Activating Token Index: 34

Text #11

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0487. Min Act: -0.0000

Data Index: 850402 (The Pile)

Max Activating Token Index: 118

Text #12

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0483. Min Act: -0.0000

Data Index: 279910 (The Pile)

Max Activating Token Index: 68

Text #13

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0504. Min Act: 0.0000

Data Index: 1211265 (The Pile)

Max Activating Token Index: 100

Text #14

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0483. Min Act: 0.0000

Data Index: 165877 (The Pile)

Max Activating Token Index: 66

Text #15

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0498. Min Act: 0.0000

Data Index: 1253568 (The Pile)

Max Activating Token Index: 393

Text #16

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0478. Min Act: 0.0000

Data Index: 1215866 (The Pile)

Max Activating Token Index: 273

Text #17

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0478. Min Act: 0.0000

Data Index: 887646 (The Pile)

Max Activating Token Index: 47

Text #18

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0476. Min Act: 0.0000

Data Index: 1475424 (The Pile)

Max Activating Token Index: 28

Text #19

Max Range: 0.0673. Min Range: -0.0673

Max Act: 0.0472. Min Act: -0.0000

Data Index: 181794 (The Pile)

Max Activating Token Index: 3