Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2381 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 8.3961. Min Range: -8.3961

Max Act: 8.3961. Min Act: -0.0000

Data Index: 817578 (The Pile)

Max Activating Token Index: 537

Text #1

Max Range: 8.3961. Min Range: -8.3961

Max Act: 8.1795. Min Act: -0.0000

Data Index: 1535760 (The Pile)

Max Activating Token Index: 323

Text #2

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.9593. Min Act: -0.0000

Data Index: 910552 (The Pile)

Max Activating Token Index: 1019

Text #3

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.7122. Min Act: -0.0000

Data Index: 194044 (The Pile)

Max Activating Token Index: 779

Text #4

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.7121. Min Act: -0.0000

Data Index: 1716295 (The Pile)

Max Activating Token Index: 688

Text #5

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.5226. Min Act: -0.0000

Data Index: 1436659 (The Pile)

Max Activating Token Index: 208

Text #6

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.5713. Min Act: -0.0000

Data Index: 300379 (The Pile)

Max Activating Token Index: 263

Text #7

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.7260. Min Act: -0.0000

Data Index: 806450 (The Pile)

Max Activating Token Index: 844

Text #8

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.1553. Min Act: -0.0000

Data Index: 1641042 (The Pile)

Max Activating Token Index: 657

Text #9

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.7311. Min Act: -0.0000

Data Index: 224142 (The Pile)

Max Activating Token Index: 718

Text #10

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.7069. Min Act: -0.0000

Data Index: 597843 (The Pile)

Max Activating Token Index: 612

Text #11

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.4829. Min Act: -0.0000

Data Index: 510146 (The Pile)

Max Activating Token Index: 269

Text #12

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.2765. Min Act: -0.0000

Data Index: 109338 (The Pile)

Max Activating Token Index: 594

Text #13

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.4380. Min Act: -0.0000

Data Index: 1452376 (The Pile)

Max Activating Token Index: 250

Text #14

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.4055. Min Act: -0.0000

Data Index: 1751331 (The Pile)

Max Activating Token Index: 750

Text #15

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.4095. Min Act: -0.0000

Data Index: 961065 (The Pile)

Max Activating Token Index: 282

Text #16

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.1988. Min Act: -0.0000

Data Index: 1804999 (The Pile)

Max Activating Token Index: 277

Text #17

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.3768. Min Act: -0.0000

Data Index: 1352707 (The Pile)

Max Activating Token Index: 178

Text #18

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.3626. Min Act: -0.0000

Data Index: 1598687 (The Pile)

Max Activating Token Index: 192

Text #19

Max Range: 8.3961. Min Range: -8.3961

Max Act: 7.0159. Min Act: -0.0000

Data Index: 597499 (The Pile)

Max Activating Token Index: 998