Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4315 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.7924. Min Act: -0.0000

Data Index: 781575 (The Pile)

Max Activating Token Index: 844

Text #1

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.7014. Min Act: -0.0000

Data Index: 1690773 (The Pile)

Max Activating Token Index: 846

Text #2

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.5973. Min Act: -0.0000

Data Index: 304739 (The Pile)

Max Activating Token Index: 911

Text #3

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4959. Min Act: -0.0000

Data Index: 841799 (The Pile)

Max Activating Token Index: 878

Text #4

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4498. Min Act: -0.0000

Data Index: 542272 (The Pile)

Max Activating Token Index: 978

Text #5

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.5089. Min Act: -0.0000

Data Index: 1600105 (The Pile)

Max Activating Token Index: 931

Text #6

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4384. Min Act: -0.0000

Data Index: 1839249 (The Pile)

Max Activating Token Index: 796

Text #7

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3801. Min Act: -0.0000

Data Index: 1101561 (The Pile)

Max Activating Token Index: 992

Text #8

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4871. Min Act: -0.0000

Data Index: 583041 (The Pile)

Max Activating Token Index: 961

Text #9

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3406. Min Act: -0.0000

Data Index: 417491 (The Pile)

Max Activating Token Index: 422

Text #10

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4540. Min Act: -0.0000

Data Index: 1652065 (The Pile)

Max Activating Token Index: 950

Text #11

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.4019. Min Act: -0.0000

Data Index: 152025 (The Pile)

Max Activating Token Index: 1006

Text #12

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3118. Min Act: -0.0000

Data Index: 784068 (The Pile)

Max Activating Token Index: 734

Text #13

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3384. Min Act: -0.0000

Data Index: 798283 (The Pile)

Max Activating Token Index: 423

Text #14

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3032. Min Act: -0.0000

Data Index: 1373415 (The Pile)

Max Activating Token Index: 978

Text #15

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3004. Min Act: -0.0000

Data Index: 178400 (The Pile)

Max Activating Token Index: 559

Text #16

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3313. Min Act: -0.0000

Data Index: 1163981 (The Pile)

Max Activating Token Index: 977

Text #17

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3230. Min Act: -0.0000

Data Index: 504638 (The Pile)

Max Activating Token Index: 144

Text #18

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.2337. Min Act: -0.0000

Data Index: 1721782 (The Pile)

Max Activating Token Index: 865

Text #19

Max Range: 1.7924. Min Range: -1.7924

Max Act: 1.3479. Min Act: -0.0000

Data Index: 128087 (The Pile)

Max Activating Token Index: 632