Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1122 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0903. Min Range: -1.0903

Max Act: 1.0903. Min Act: -0.0000

Data Index: 1585807 (The Pile)

Max Activating Token Index: 892

Text #1

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.7569. Min Act: -0.0000

Data Index: 1264548 (The Pile)

Max Activating Token Index: 506

Text #2

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.6637. Min Act: -0.0000

Data Index: 1034100 (The Pile)

Max Activating Token Index: 345

Text #3

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5919. Min Act: -0.0000

Data Index: 1869686 (The Pile)

Max Activating Token Index: 112

Text #4

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5786. Min Act: -0.0000

Data Index: 215420 (The Pile)

Max Activating Token Index: 876

Text #5

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.6635. Min Act: -0.0000

Data Index: 857762 (The Pile)

Max Activating Token Index: 321

Text #6

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.6635. Min Act: -0.0000

Data Index: 217758 (The Pile)

Max Activating Token Index: 859

Text #7

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.6152. Min Act: -0.0000

Data Index: 433407 (The Pile)

Max Activating Token Index: 413

Text #8

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5471. Min Act: -0.0000

Data Index: 637460 (The Pile)

Max Activating Token Index: 548

Text #9

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.6366. Min Act: -0.0000

Data Index: 1036965 (The Pile)

Max Activating Token Index: 534

Text #10

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5639. Min Act: -0.0000

Data Index: 1905148 (The Pile)

Max Activating Token Index: 308

Text #11

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5841. Min Act: -0.0000

Data Index: 201432 (The Pile)

Max Activating Token Index: 277

Text #12

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5252. Min Act: -0.0000

Data Index: 568860 (The Pile)

Max Activating Token Index: 298

Text #13

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5540. Min Act: -0.0000

Data Index: 1893514 (The Pile)

Max Activating Token Index: 170

Text #14

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5564. Min Act: -0.0000

Data Index: 1539905 (The Pile)

Max Activating Token Index: 439

Text #15

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5355. Min Act: -0.0000

Data Index: 702888 (The Pile)

Max Activating Token Index: 659

Text #16

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5732. Min Act: -0.0000

Data Index: 995757 (The Pile)

Max Activating Token Index: 183

Text #17

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.4536. Min Act: -0.0000

Data Index: 1446883 (The Pile)

Max Activating Token Index: 978

Text #18

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5496. Min Act: -0.0000

Data Index: 1623706 (The Pile)

Max Activating Token Index: 533

Text #19

Max Range: 1.0903. Min Range: -1.0903

Max Act: 0.5476. Min Act: -0.0000

Data Index: 1438559 (The Pile)

Max Activating Token Index: 274