Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2264 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.1062. Min Act: -0.0000

Data Index: 272761 (The Pile)

Max Activating Token Index: 52

Text #1

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.1019. Min Act: -0.0000

Data Index: 667648 (The Pile)

Max Activating Token Index: 61

Text #2

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0918. Min Act: -0.0000

Data Index: 694438 (The Pile)

Max Activating Token Index: 1023

Text #3

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0855. Min Act: -0.0000

Data Index: 424626 (The Pile)

Max Activating Token Index: 423

Text #4

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0864. Min Act: -0.0000

Data Index: 462980 (The Pile)

Max Activating Token Index: 47

Text #5

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0819. Min Act: -0.0000

Data Index: 1538346 (The Pile)

Max Activating Token Index: 44

Text #6

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0811. Min Act: -0.0000

Data Index: 1405701 (The Pile)

Max Activating Token Index: 812

Text #7

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0792. Min Act: -0.0000

Data Index: 237809 (The Pile)

Max Activating Token Index: 146

Text #8

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0794. Min Act: -0.0000

Data Index: 1781620 (The Pile)

Max Activating Token Index: 445

Text #9

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0792. Min Act: -0.0000

Data Index: 482994 (The Pile)

Max Activating Token Index: 240

Text #10

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0775. Min Act: -0.0000

Data Index: 1773722 (The Pile)

Max Activating Token Index: 109

Text #11

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0761. Min Act: -0.0000

Data Index: 1866815 (The Pile)

Max Activating Token Index: 973

Text #12

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0753. Min Act: -0.0000

Data Index: 1164608 (The Pile)

Max Activating Token Index: 146

Text #13

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0751. Min Act: -0.0000

Data Index: 1255206 (The Pile)

Max Activating Token Index: 38

Text #14

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0751. Min Act: -0.0000

Data Index: 1941619 (The Pile)

Max Activating Token Index: 142

Text #15

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0748. Min Act: -0.0000

Data Index: 1921065 (The Pile)

Max Activating Token Index: 130

Text #16

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0744. Min Act: -0.0000

Data Index: 609017 (The Pile)

Max Activating Token Index: 337

Text #17

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0745. Min Act: -0.0000

Data Index: 1022315 (The Pile)

Max Activating Token Index: 683

Text #18

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0737. Min Act: -0.0000

Data Index: 658513 (The Pile)

Max Activating Token Index: 561

Text #19

Max Range: 0.1062. Min Range: -0.1062

Max Act: 0.0738. Min Act: -0.0000

Data Index: 990156 (The Pile)

Max Activating Token Index: 303