Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 956 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.3000. Min Act: -0.0000

Data Index: 1947346 (The Pile)

Max Activating Token Index: 86

Text #1

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.1772. Min Act: -0.0000

Data Index: 200384 (The Pile)

Max Activating Token Index: 672

Text #2

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.2298. Min Act: -0.0000

Data Index: 1971058 (The Pile)

Max Activating Token Index: 253

Text #3

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.1708. Min Act: -0.0000

Data Index: 1364733 (The Pile)

Max Activating Token Index: 100

Text #4

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0787. Min Act: -0.0000

Data Index: 431154 (The Pile)

Max Activating Token Index: 772

Text #5

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0742. Min Act: -0.0000

Data Index: 1629304 (The Pile)

Max Activating Token Index: 17

Text #6

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0179. Min Act: -0.0000

Data Index: 1835422 (The Pile)

Max Activating Token Index: 884

Text #7

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0484. Min Act: -0.0000

Data Index: 1353120 (The Pile)

Max Activating Token Index: 4

Text #8

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0857. Min Act: -0.0000

Data Index: 1132718 (The Pile)

Max Activating Token Index: 6

Text #9

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0424. Min Act: -0.0000

Data Index: 223382 (The Pile)

Max Activating Token Index: 121

Text #10

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0478. Min Act: -0.0000

Data Index: 712539 (The Pile)

Max Activating Token Index: 23

Text #11

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0226. Min Act: -0.0000

Data Index: 1963711 (The Pile)

Max Activating Token Index: 630

Text #12

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0391. Min Act: -0.0000

Data Index: 1711130 (The Pile)

Max Activating Token Index: 857

Text #13

Max Range: 1.3000. Min Range: -1.3000

Max Act: 0.9462. Min Act: -0.0000

Data Index: 957794 (The Pile)

Max Activating Token Index: 33

Text #14

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0034. Min Act: -0.0000

Data Index: 1214837 (The Pile)

Max Activating Token Index: 656

Text #15

Max Range: 1.3000. Min Range: -1.3000

Max Act: 1.0474. Min Act: -0.0000

Data Index: 1769472 (The Pile)

Max Activating Token Index: 18

Text #16

Max Range: 1.3000. Min Range: -1.3000

Max Act: 0.9933. Min Act: -0.0000

Data Index: 85926 (The Pile)

Max Activating Token Index: 519

Text #17

Max Range: 1.3000. Min Range: -1.3000

Max Act: 0.9977. Min Act: -0.0000

Data Index: 1999175 (The Pile)

Max Activating Token Index: 512

Text #18

Max Range: 1.3000. Min Range: -1.3000

Max Act: 0.9523. Min Act: -0.0000

Data Index: 415890 (The Pile)

Max Activating Token Index: 880

Text #19

Max Range: 1.3000. Min Range: -1.3000

Max Act: 0.9833. Min Act: -0.0000

Data Index: 682722 (The Pile)

Max Activating Token Index: 963