Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 946 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.5294. Min Act: -0.0000

Data Index: 1260459 (The Pile)

Max Activating Token Index: 877

Text #1

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.5542. Min Act: -0.0000

Data Index: 907361 (The Pile)

Max Activating Token Index: 354

Text #2

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4767. Min Act: -0.0000

Data Index: 1763837 (The Pile)

Max Activating Token Index: 260

Text #3

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4794. Min Act: -0.0000

Data Index: 294107 (The Pile)

Max Activating Token Index: 217

Text #4

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4578. Min Act: -0.0000

Data Index: 293217 (The Pile)

Max Activating Token Index: 512

Text #5

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4373. Min Act: -0.0000

Data Index: 982111 (The Pile)

Max Activating Token Index: 638

Text #6

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4290. Min Act: -0.0000

Data Index: 1829796 (The Pile)

Max Activating Token Index: 243

Text #7

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4299. Min Act: -0.0000

Data Index: 92322 (The Pile)

Max Activating Token Index: 589

Text #8

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4436. Min Act: -0.0000

Data Index: 1752404 (The Pile)

Max Activating Token Index: 770

Text #9

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4219. Min Act: -0.0000

Data Index: 1341733 (The Pile)

Max Activating Token Index: 546

Text #10

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4253. Min Act: -0.0000

Data Index: 685215 (The Pile)

Max Activating Token Index: 433

Text #11

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4260. Min Act: -0.0000

Data Index: 1301046 (The Pile)

Max Activating Token Index: 136

Text #12

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.3987. Min Act: -0.0000

Data Index: 606591 (The Pile)

Max Activating Token Index: 702

Text #13

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4009. Min Act: -0.0000

Data Index: 116582 (The Pile)

Max Activating Token Index: 817

Text #14

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4252. Min Act: -0.0000

Data Index: 263158 (The Pile)

Max Activating Token Index: 444

Text #15

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.3895. Min Act: 0.0000

Data Index: 46088 (The Pile)

Max Activating Token Index: 332

Text #16

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.3792. Min Act: -0.0000

Data Index: 1981051 (The Pile)

Max Activating Token Index: 538

Text #17

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4224. Min Act: -0.0000

Data Index: 1486551 (The Pile)

Max Activating Token Index: 136

Text #18

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.3940. Min Act: -0.0000

Data Index: 1814832 (The Pile)

Max Activating Token Index: 244

Text #19

Max Range: 0.5542. Min Range: -0.5542

Max Act: 0.4206. Min Act: -0.0000

Data Index: 587428 (The Pile)

Max Activating Token Index: 935