Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 362 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.1154. Min Act: -0.0000

Data Index: 1332607 (The Pile)

Max Activating Token Index: 557

Text #1

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.1271. Min Act: -0.0000

Data Index: 1823743 (The Pile)

Max Activating Token Index: 442

Text #2

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0886. Min Act: -0.0000

Data Index: 1107357 (The Pile)

Max Activating Token Index: 225

Text #3

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0594. Min Act: -0.0000

Data Index: 1188985 (The Pile)

Max Activating Token Index: 82

Text #4

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0411. Min Act: -0.0000

Data Index: 1733786 (The Pile)

Max Activating Token Index: 326

Text #5

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0305. Min Act: -0.0000

Data Index: 1260873 (The Pile)

Max Activating Token Index: 675

Text #6

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0624. Min Act: -0.0000

Data Index: 956074 (The Pile)

Max Activating Token Index: 33

Text #7

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0827. Min Act: -0.0000

Data Index: 633142 (The Pile)

Max Activating Token Index: 41

Text #8

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9667. Min Act: -0.0000

Data Index: 1048341 (The Pile)

Max Activating Token Index: 879

Text #9

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9953. Min Act: -0.0000

Data Index: 99876 (The Pile)

Max Activating Token Index: 168

Text #10

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9897. Min Act: -0.0000

Data Index: 768036 (The Pile)

Max Activating Token Index: 31

Text #11

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0466. Min Act: -0.0000

Data Index: 462493 (The Pile)

Max Activating Token Index: 150

Text #12

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9571. Min Act: -0.0000

Data Index: 1033038 (The Pile)

Max Activating Token Index: 79

Text #13

Max Range: 1.1271. Min Range: -1.1271

Max Act: 1.0379. Min Act: -0.0000

Data Index: 573935 (The Pile)

Max Activating Token Index: 119

Text #14

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9573. Min Act: -0.0000

Data Index: 1903007 (The Pile)

Max Activating Token Index: 546

Text #15

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9893. Min Act: -0.0000

Data Index: 1131372 (The Pile)

Max Activating Token Index: 558

Text #16

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9389. Min Act: -0.0000

Data Index: 1339731 (The Pile)

Max Activating Token Index: 169

Text #17

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9846. Min Act: -0.0000

Data Index: 1784651 (The Pile)

Max Activating Token Index: 700

Text #18

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.9551. Min Act: -0.0000

Data Index: 1106757 (The Pile)

Max Activating Token Index: 194

Text #19

Max Range: 1.1271. Min Range: -1.1271

Max Act: 0.8988. Min Act: -0.0000

Data Index: 1963499 (The Pile)

Max Activating Token Index: 876