Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4629 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.8010. Min Act: -0.0000

Data Index: 292362 (The Pile)

Max Activating Token Index: 749

Text #1

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.5803. Min Act: -0.0000

Data Index: 1390712 (The Pile)

Max Activating Token Index: 103

Text #2

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.4626. Min Act: -0.0000

Data Index: 174586 (The Pile)

Max Activating Token Index: 21

Text #3

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.3118. Min Act: -0.0000

Data Index: 1058359 (The Pile)

Max Activating Token Index: 51

Text #4

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.3394. Min Act: -0.0000

Data Index: 1924129 (The Pile)

Max Activating Token Index: 695

Text #5

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.2573. Min Act: -0.0000

Data Index: 1324467 (The Pile)

Max Activating Token Index: 719

Text #6

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1413. Min Act: -0.0000

Data Index: 1176612 (The Pile)

Max Activating Token Index: 160

Text #7

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.2421. Min Act: -0.0000

Data Index: 431245 (The Pile)

Max Activating Token Index: 473

Text #8

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.2284. Min Act: -0.0000

Data Index: 819946 (The Pile)

Max Activating Token Index: 851

Text #9

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1784. Min Act: -0.0000

Data Index: 1082138 (The Pile)

Max Activating Token Index: 332

Text #10

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1977. Min Act: -0.0000

Data Index: 896753 (The Pile)

Max Activating Token Index: 361

Text #11

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.2786. Min Act: -0.0000

Data Index: 7825 (The Pile)

Max Activating Token Index: 860

Text #12

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1876. Min Act: -0.0000

Data Index: 266048 (The Pile)

Max Activating Token Index: 147

Text #13

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.0908. Min Act: -0.0000

Data Index: 1624888 (The Pile)

Max Activating Token Index: 191

Text #14

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1515. Min Act: -0.0000

Data Index: 1383339 (The Pile)

Max Activating Token Index: 127

Text #15

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1777. Min Act: -0.0000

Data Index: 1376503 (The Pile)

Max Activating Token Index: 479

Text #16

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1355. Min Act: -0.0000

Data Index: 1759671 (The Pile)

Max Activating Token Index: 776

Text #17

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.2193. Min Act: -0.0000

Data Index: 959370 (The Pile)

Max Activating Token Index: 641

Text #18

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.1015. Min Act: -0.0000

Data Index: 1276906 (The Pile)

Max Activating Token Index: 219

Text #19

Max Range: 1.8010. Min Range: -1.8010

Max Act: 1.0916. Min Act: -0.0000

Data Index: 1666333 (The Pile)

Max Activating Token Index: 650