Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2425 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.9299. Min Act: -0.0000

Data Index: 1430641 (The Pile)

Max Activating Token Index: 309

Text #1

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.9379. Min Act: -0.0000

Data Index: 1289899 (The Pile)

Max Activating Token Index: 293

Text #2

Max Range: 3.0629. Min Range: -3.0629

Max Act: 3.0629. Min Act: -0.0000

Data Index: 157344 (The Pile)

Max Activating Token Index: 212

Text #3

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.9114. Min Act: -0.0000

Data Index: 1493870 (The Pile)

Max Activating Token Index: 32

Text #4

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.6600. Min Act: -0.0000

Data Index: 782119 (The Pile)

Max Activating Token Index: 37

Text #5

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.7678. Min Act: -0.0000

Data Index: 128726 (The Pile)

Max Activating Token Index: 179

Text #6

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.4568. Min Act: -0.0000

Data Index: 217038 (The Pile)

Max Activating Token Index: 954

Text #7

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.2785. Min Act: -0.0000

Data Index: 783108 (The Pile)

Max Activating Token Index: 946

Text #8

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.3209. Min Act: -0.0000

Data Index: 1246159 (The Pile)

Max Activating Token Index: 833

Text #9

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.4667. Min Act: -0.0000

Data Index: 818031 (The Pile)

Max Activating Token Index: 854

Text #10

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.1956. Min Act: -0.0000

Data Index: 316242 (The Pile)

Max Activating Token Index: 107

Text #11

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.0516. Min Act: -0.0000

Data Index: 1524667 (The Pile)

Max Activating Token Index: 578

Text #12

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.4272. Min Act: -0.0000

Data Index: 324302 (The Pile)

Max Activating Token Index: 560

Text #13

Max Range: 3.0629. Min Range: -3.0629

Max Act: 1.9163. Min Act: -0.0000

Data Index: 620149 (The Pile)

Max Activating Token Index: 76

Text #14

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.0922. Min Act: -0.0000

Data Index: 187385 (The Pile)

Max Activating Token Index: 193

Text #15

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.6683. Min Act: -0.0000

Data Index: 1734251 (The Pile)

Max Activating Token Index: 753

Text #16

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.1902. Min Act: -0.0000

Data Index: 560936 (The Pile)

Max Activating Token Index: 619

Text #17

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.1791. Min Act: -0.0000

Data Index: 328881 (The Pile)

Max Activating Token Index: 531

Text #18

Max Range: 3.0629. Min Range: -3.0629

Max Act: 2.0539. Min Act: -0.0000

Data Index: 131166 (The Pile)

Max Activating Token Index: 709

Text #19

Max Range: 3.0629. Min Range: -3.0629

Max Act: 1.9310. Min Act: -0.0000

Data Index: 213990 (The Pile)

Max Activating Token Index: 692