Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1456 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.6493. Min Act: -0.0000

Data Index: 974552 (The Pile)

Max Activating Token Index: 209

Text #1

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.6168. Min Act: -0.0000

Data Index: 1597070 (The Pile)

Max Activating Token Index: 449

Text #2

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.5416. Min Act: -0.0000

Data Index: 106942 (The Pile)

Max Activating Token Index: 169

Text #3

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.6214. Min Act: -0.0000

Data Index: 1655111 (The Pile)

Max Activating Token Index: 317

Text #4

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.4649. Min Act: -0.0000

Data Index: 1875007 (The Pile)

Max Activating Token Index: 66

Text #5

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.4150. Min Act: -0.0000

Data Index: 532559 (The Pile)

Max Activating Token Index: 787

Text #6

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.4026. Min Act: -0.0000

Data Index: 1715139 (The Pile)

Max Activating Token Index: 302

Text #7

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3490. Min Act: -0.0000

Data Index: 452622 (The Pile)

Max Activating Token Index: 191

Text #8

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3687. Min Act: -0.0000

Data Index: 1040917 (The Pile)

Max Activating Token Index: 479

Text #9

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3991. Min Act: -0.0000

Data Index: 1488058 (The Pile)

Max Activating Token Index: 315

Text #10

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3485. Min Act: -0.0000

Data Index: 110832 (The Pile)

Max Activating Token Index: 887

Text #11

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.2912. Min Act: -0.0000

Data Index: 1909550 (The Pile)

Max Activating Token Index: 191

Text #12

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3691. Min Act: -0.0000

Data Index: 1866620 (The Pile)

Max Activating Token Index: 207

Text #13

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3305. Min Act: -0.0000

Data Index: 849642 (The Pile)

Max Activating Token Index: 121

Text #14

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3222. Min Act: -0.0000

Data Index: 941955 (The Pile)

Max Activating Token Index: 207

Text #15

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3426. Min Act: -0.0000

Data Index: 607593 (The Pile)

Max Activating Token Index: 172

Text #16

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3111. Min Act: -0.0000

Data Index: 1408203 (The Pile)

Max Activating Token Index: 421

Text #17

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.2720. Min Act: -0.0000

Data Index: 1000599 (The Pile)

Max Activating Token Index: 516

Text #18

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.3030. Min Act: -0.0000

Data Index: 1274558 (The Pile)

Max Activating Token Index: 586

Text #19

Max Range: 1.6493. Min Range: -1.6493

Max Act: 1.2633. Min Act: -0.0000

Data Index: 1365326 (The Pile)

Max Activating Token Index: 539