Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 668 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2691. Min Range: -1.2691

Max Act: 1.2691. Min Act: -0.0000

Data Index: 1448584 (The Pile)

Max Activating Token Index: 400

Text #1

Max Range: 1.2691. Min Range: -1.2691

Max Act: 1.1047. Min Act: -0.0000

Data Index: 324691 (The Pile)

Max Activating Token Index: 1003

Text #2

Max Range: 1.2691. Min Range: -1.2691

Max Act: 1.1425. Min Act: -0.0000

Data Index: 1714562 (The Pile)

Max Activating Token Index: 641

Text #3

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.9779. Min Act: -0.0000

Data Index: 1501123 (The Pile)

Max Activating Token Index: 205

Text #4

Max Range: 1.2691. Min Range: -1.2691

Max Act: 1.0075. Min Act: -0.0000

Data Index: 998651 (The Pile)

Max Activating Token Index: 193

Text #5

Max Range: 1.2691. Min Range: -1.2691

Max Act: 1.0053. Min Act: -0.0000

Data Index: 1316107 (The Pile)

Max Activating Token Index: 668

Text #6

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.9052. Min Act: -0.0000

Data Index: 992385 (The Pile)

Max Activating Token Index: 79

Text #7

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8871. Min Act: -0.0000

Data Index: 591325 (The Pile)

Max Activating Token Index: 409

Text #8

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.9060. Min Act: -0.0000

Data Index: 728467 (The Pile)

Max Activating Token Index: 556

Text #9

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.9834. Min Act: -0.0000

Data Index: 1518128 (The Pile)

Max Activating Token Index: 396

Text #10

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8828. Min Act: -0.0000

Data Index: 1369103 (The Pile)

Max Activating Token Index: 306

Text #11

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8214. Min Act: -0.0000

Data Index: 1822700 (The Pile)

Max Activating Token Index: 587

Text #12

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8467. Min Act: -0.0000

Data Index: 1058645 (The Pile)

Max Activating Token Index: 243

Text #13

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8492. Min Act: -0.0000

Data Index: 629883 (The Pile)

Max Activating Token Index: 387

Text #14

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.7962. Min Act: -0.0000

Data Index: 1817070 (The Pile)

Max Activating Token Index: 598

Text #15

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.7835. Min Act: -0.0000

Data Index: 1097230 (The Pile)

Max Activating Token Index: 964

Text #16

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.7796. Min Act: -0.0000

Data Index: 1262697 (The Pile)

Max Activating Token Index: 471

Text #17

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8241. Min Act: -0.0000

Data Index: 1911641 (The Pile)

Max Activating Token Index: 281

Text #18

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.7937. Min Act: -0.0000

Data Index: 1004661 (The Pile)

Max Activating Token Index: 494

Text #19

Max Range: 1.2691. Min Range: -1.2691

Max Act: 0.8428. Min Act: -0.0000

Data Index: 1061128 (The Pile)

Max Activating Token Index: 230