Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3439 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.6456. Min Act: -0.0000

Data Index: 802571 (The Pile)

Max Activating Token Index: 986

Text #1

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.3364. Min Act: -0.0000

Data Index: 850970 (The Pile)

Max Activating Token Index: 1008

Text #2

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.0852. Min Act: -0.0000

Data Index: 776874 (The Pile)

Max Activating Token Index: 1010

Text #3

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.1720. Min Act: -0.0000

Data Index: 1306351 (The Pile)

Max Activating Token Index: 148

Text #4

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.9901. Min Act: -0.0000

Data Index: 1931530 (The Pile)

Max Activating Token Index: 812

Text #5

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.1086. Min Act: -0.0000

Data Index: 251911 (The Pile)

Max Activating Token Index: 874

Text #6

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.0056. Min Act: -0.0000

Data Index: 82863 (The Pile)

Max Activating Token Index: 41

Text #7

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.0065. Min Act: -0.0000

Data Index: 1991772 (The Pile)

Max Activating Token Index: 475

Text #8

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.0011. Min Act: -0.0000

Data Index: 1827592 (The Pile)

Max Activating Token Index: 729

Text #9

Max Range: 3.6456. Min Range: -3.6456

Max Act: 3.1601. Min Act: -0.0000

Data Index: 565150 (The Pile)

Max Activating Token Index: 603

Text #10

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.9481. Min Act: -0.0000

Data Index: 996317 (The Pile)

Max Activating Token Index: 935

Text #11

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.9639. Min Act: -0.0000

Data Index: 1726314 (The Pile)

Max Activating Token Index: 564

Text #12

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.9124. Min Act: -0.0000

Data Index: 1711574 (The Pile)

Max Activating Token Index: 1008

Text #13

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.8784. Min Act: -0.0000

Data Index: 645759 (The Pile)

Max Activating Token Index: 106

Text #14

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.8530. Min Act: -0.0000

Data Index: 1566212 (The Pile)

Max Activating Token Index: 816

Text #15

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.7101. Min Act: -0.0000

Data Index: 1445974 (The Pile)

Max Activating Token Index: 32

Text #16

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.7641. Min Act: -0.0000

Data Index: 1366911 (The Pile)

Max Activating Token Index: 357

Text #17

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.7621. Min Act: -0.0000

Data Index: 223327 (The Pile)

Max Activating Token Index: 150

Text #18

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.7621. Min Act: -0.0000

Data Index: 1375867 (The Pile)

Max Activating Token Index: 259

Text #19

Max Range: 3.6456. Min Range: -3.6456

Max Act: 2.9033. Min Act: -0.0000

Data Index: 738761 (The Pile)

Max Activating Token Index: 433