Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1271 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.6027. Min Act: -0.0000

Data Index: 1815014 (The Pile)

Max Activating Token Index: 119

Text #1

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4968. Min Act: -0.0000

Data Index: 1011783 (The Pile)

Max Activating Token Index: 588

Text #2

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.5231. Min Act: -0.0000

Data Index: 73666 (The Pile)

Max Activating Token Index: 565

Text #3

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4847. Min Act: -0.0000

Data Index: 232050 (The Pile)

Max Activating Token Index: 620

Text #4

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4964. Min Act: -0.0000

Data Index: 325442 (The Pile)

Max Activating Token Index: 862

Text #5

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4591. Min Act: 0.0000

Data Index: 764246 (The Pile)

Max Activating Token Index: 873

Text #6

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4777. Min Act: -0.0000

Data Index: 1231448 (The Pile)

Max Activating Token Index: 368

Text #7

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4438. Min Act: -0.0000

Data Index: 450134 (The Pile)

Max Activating Token Index: 824

Text #8

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4345. Min Act: -0.0000

Data Index: 299912 (The Pile)

Max Activating Token Index: 997

Text #9

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4229. Min Act: -0.0000

Data Index: 1971452 (The Pile)

Max Activating Token Index: 385

Text #10

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4353. Min Act: -0.0000

Data Index: 529646 (The Pile)

Max Activating Token Index: 186

Text #11

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4418. Min Act: -0.0000

Data Index: 334326 (The Pile)

Max Activating Token Index: 225

Text #12

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4377. Min Act: -0.0000

Data Index: 146414 (The Pile)

Max Activating Token Index: 204

Text #13

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4136. Min Act: -0.0000

Data Index: 1861747 (The Pile)

Max Activating Token Index: 690

Text #14

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4104. Min Act: -0.0000

Data Index: 1987578 (The Pile)

Max Activating Token Index: 869

Text #15

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.3970. Min Act: -0.0000

Data Index: 1494016 (The Pile)

Max Activating Token Index: 1004

Text #16

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4250. Min Act: -0.0000

Data Index: 643991 (The Pile)

Max Activating Token Index: 137

Text #17

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4116. Min Act: 0.0000

Data Index: 1020044 (The Pile)

Max Activating Token Index: 841

Text #18

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.3996. Min Act: -0.0000

Data Index: 452541 (The Pile)

Max Activating Token Index: 496

Text #19

Max Range: 0.6027. Min Range: -0.6027

Max Act: 0.4008. Min Act: -0.0000

Data Index: 1634021 (The Pile)

Max Activating Token Index: 414