Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 411 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.4597. Min Act: -0.0000

Data Index: 646001 (The Pile)

Max Activating Token Index: 461

Text #1

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.3211. Min Act: -0.0000

Data Index: 776120 (The Pile)

Max Activating Token Index: 315

Text #2

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.4613. Min Act: -0.0000

Data Index: 723107 (The Pile)

Max Activating Token Index: 286

Text #3

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.3853. Min Act: -0.0000

Data Index: 236194 (The Pile)

Max Activating Token Index: 266

Text #4

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.2468. Min Act: -0.0000

Data Index: 33814 (The Pile)

Max Activating Token Index: 502

Text #5

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.2540. Min Act: -0.0000

Data Index: 96327 (The Pile)

Max Activating Token Index: 536

Text #6

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.1113. Min Act: -0.0000

Data Index: 846447 (The Pile)

Max Activating Token Index: 175

Text #7

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.0634. Min Act: -0.0000

Data Index: 1859116 (The Pile)

Max Activating Token Index: 965

Text #8

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.0375. Min Act: -0.0000

Data Index: 527317 (The Pile)

Max Activating Token Index: 861

Text #9

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.9500. Min Act: -0.0000

Data Index: 1589662 (The Pile)

Max Activating Token Index: 607

Text #10

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.0444. Min Act: -0.0000

Data Index: 1450878 (The Pile)

Max Activating Token Index: 682

Text #11

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.9002. Min Act: -0.0000

Data Index: 1747829 (The Pile)

Max Activating Token Index: 328

Text #12

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.8809. Min Act: -0.0000

Data Index: 860350 (The Pile)

Max Activating Token Index: 715

Text #13

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.8419. Min Act: -0.0000

Data Index: 694640 (The Pile)

Max Activating Token Index: 926

Text #14

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.1098. Min Act: -0.0000

Data Index: 270858 (The Pile)

Max Activating Token Index: 249

Text #15

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.9068. Min Act: -0.0000

Data Index: 800585 (The Pile)

Max Activating Token Index: 385

Text #16

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.9069. Min Act: -0.0000

Data Index: 1681631 (The Pile)

Max Activating Token Index: 264

Text #17

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.9705. Min Act: -0.0000

Data Index: 1498865 (The Pile)

Max Activating Token Index: 472

Text #18

Max Range: 14.4613. Min Range: -14.4613

Max Act: 14.0403. Min Act: -0.0000

Data Index: 291200 (The Pile)

Max Activating Token Index: 738

Text #19

Max Range: 14.4613. Min Range: -14.4613

Max Act: 13.7572. Min Act: -0.0000

Data Index: 1985093 (The Pile)

Max Activating Token Index: 255