Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 879 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.9722. Min Act: -0.0000

Data Index: 71001 (The Pile)

Max Activating Token Index: 708

Text #1

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.8756. Min Act: -0.0000

Data Index: 904997 (The Pile)

Max Activating Token Index: 749

Text #2

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.7713. Min Act: -0.0000

Data Index: 1272853 (The Pile)

Max Activating Token Index: 78

Text #3

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5619. Min Act: -0.0000

Data Index: 1931295 (The Pile)

Max Activating Token Index: 311

Text #4

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.7339. Min Act: -0.0000

Data Index: 570585 (The Pile)

Max Activating Token Index: 99

Text #5

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.6102. Min Act: -0.0000

Data Index: 627154 (The Pile)

Max Activating Token Index: 652

Text #6

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.6055. Min Act: -0.0000

Data Index: 1270939 (The Pile)

Max Activating Token Index: 562

Text #7

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5666. Min Act: -0.0000

Data Index: 864463 (The Pile)

Max Activating Token Index: 88

Text #8

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5547. Min Act: -0.0000

Data Index: 826212 (The Pile)

Max Activating Token Index: 399

Text #9

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5416. Min Act: -0.0000

Data Index: 1763587 (The Pile)

Max Activating Token Index: 405

Text #10

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5900. Min Act: -0.0000

Data Index: 1228068 (The Pile)

Max Activating Token Index: 284

Text #11

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.4750. Min Act: -0.0000

Data Index: 152862 (The Pile)

Max Activating Token Index: 143

Text #12

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.4674. Min Act: -0.0000

Data Index: 1250794 (The Pile)

Max Activating Token Index: 425

Text #13

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5466. Min Act: -0.0000

Data Index: 696854 (The Pile)

Max Activating Token Index: 518

Text #14

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.3756. Min Act: -0.0000

Data Index: 1961791 (The Pile)

Max Activating Token Index: 31

Text #15

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.4601. Min Act: -0.0000

Data Index: 1440230 (The Pile)

Max Activating Token Index: 159

Text #16

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.4394. Min Act: -0.0000

Data Index: 277137 (The Pile)

Max Activating Token Index: 395

Text #17

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5288. Min Act: -0.0000

Data Index: 1752632 (The Pile)

Max Activating Token Index: 435

Text #18

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.4286. Min Act: -0.0000

Data Index: 870021 (The Pile)

Max Activating Token Index: 313

Text #19

Max Range: 1.9722. Min Range: -1.9722

Max Act: 1.5000. Min Act: -0.0000

Data Index: 645675 (The Pile)

Max Activating Token Index: 518