Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 563 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1955. Min Range: -3.1955

Max Act: 3.1955. Min Act: -0.0000

Data Index: 1232580 (The Pile)

Max Activating Token Index: 566

Text #1

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.2581. Min Act: -0.0000

Data Index: 1709530 (The Pile)

Max Activating Token Index: 967

Text #2

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.1071. Min Act: -0.0000

Data Index: 672468 (The Pile)

Max Activating Token Index: 438

Text #3

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.0899. Min Act: -0.0000

Data Index: 264599 (The Pile)

Max Activating Token Index: 660

Text #4

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9258. Min Act: -0.0000

Data Index: 473664 (The Pile)

Max Activating Token Index: 861

Text #5

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.1446. Min Act: -0.0000

Data Index: 438251 (The Pile)

Max Activating Token Index: 580

Text #6

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9313. Min Act: -0.0000

Data Index: 534191 (The Pile)

Max Activating Token Index: 922

Text #7

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.0529. Min Act: -0.0000

Data Index: 1153475 (The Pile)

Max Activating Token Index: 402

Text #8

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.0237. Min Act: -0.0000

Data Index: 1687586 (The Pile)

Max Activating Token Index: 665

Text #9

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9768. Min Act: -0.0000

Data Index: 195117 (The Pile)

Max Activating Token Index: 418

Text #10

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9962. Min Act: -0.0000

Data Index: 675659 (The Pile)

Max Activating Token Index: 220

Text #11

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9570. Min Act: -0.0000

Data Index: 1641606 (The Pile)

Max Activating Token Index: 267

Text #12

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9072. Min Act: -0.0000

Data Index: 517250 (The Pile)

Max Activating Token Index: 343

Text #13

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9800. Min Act: -0.0000

Data Index: 1632811 (The Pile)

Max Activating Token Index: 33

Text #14

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.9016. Min Act: -0.0000

Data Index: 313816 (The Pile)

Max Activating Token Index: 195

Text #15

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.8970. Min Act: -0.0000

Data Index: 327180 (The Pile)

Max Activating Token Index: 377

Text #16

Max Range: 3.1955. Min Range: -3.1955

Max Act: 1.1503. Min Act: -0.0000

Data Index: 1612792 (The Pile)

Max Activating Token Index: 871

Text #17

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.8579. Min Act: -0.0000

Data Index: 1530058 (The Pile)

Max Activating Token Index: 584

Text #18

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.8551. Min Act: -0.0000

Data Index: 1644566 (The Pile)

Max Activating Token Index: 201

Text #19

Max Range: 3.1955. Min Range: -3.1955

Max Act: 0.8519. Min Act: -0.0000

Data Index: 534923 (The Pile)

Max Activating Token Index: 219