Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2843 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 6.4297. Min Range: -6.4297

Max Act: 6.4297. Min Act: -0.0000

Data Index: 1740956 (The Pile)

Max Activating Token Index: 410

Text #1

Max Range: 6.4297. Min Range: -6.4297

Max Act: 6.1503. Min Act: -0.0000

Data Index: 41134 (The Pile)

Max Activating Token Index: 283

Text #2

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.7893. Min Act: -0.0000

Data Index: 1008000 (The Pile)

Max Activating Token Index: 346

Text #3

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.7614. Min Act: -0.0000

Data Index: 1249374 (The Pile)

Max Activating Token Index: 391

Text #4

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.8798. Min Act: -0.0000

Data Index: 256234 (The Pile)

Max Activating Token Index: 223

Text #5

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.3778. Min Act: -0.0000

Data Index: 1617712 (The Pile)

Max Activating Token Index: 741

Text #6

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.4110. Min Act: -0.0000

Data Index: 121652 (The Pile)

Max Activating Token Index: 479

Text #7

Max Range: 6.4297. Min Range: -6.4297

Max Act: 6.0309. Min Act: -0.0000

Data Index: 941706 (The Pile)

Max Activating Token Index: 583

Text #8

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.2355. Min Act: -0.0000

Data Index: 720871 (The Pile)

Max Activating Token Index: 203

Text #9

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.6665. Min Act: -0.0000

Data Index: 256899 (The Pile)

Max Activating Token Index: 473

Text #10

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.2620. Min Act: -0.0000

Data Index: 320025 (The Pile)

Max Activating Token Index: 524

Text #11

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.4983. Min Act: -0.0000

Data Index: 1714025 (The Pile)

Max Activating Token Index: 536

Text #12

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.0419. Min Act: -0.0000

Data Index: 1449874 (The Pile)

Max Activating Token Index: 522

Text #13

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.8728. Min Act: -0.0000

Data Index: 1598507 (The Pile)

Max Activating Token Index: 627

Text #14

Max Range: 6.4297. Min Range: -6.4297

Max Act: 5.2333. Min Act: -0.0000

Data Index: 1119809 (The Pile)

Max Activating Token Index: 348

Text #15

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.9750. Min Act: -0.0000

Data Index: 1936373 (The Pile)

Max Activating Token Index: 610

Text #16

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.6076. Min Act: -0.0000

Data Index: 1599726 (The Pile)

Max Activating Token Index: 323

Text #17

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.8111. Min Act: -0.0000

Data Index: 1434527 (The Pile)

Max Activating Token Index: 474

Text #18

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.6707. Min Act: -0.0000

Data Index: 1003715 (The Pile)

Max Activating Token Index: 178

Text #19

Max Range: 6.4297. Min Range: -6.4297

Max Act: 4.8958. Min Act: -0.0000

Data Index: 1837419 (The Pile)

Max Activating Token Index: 759