Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2418 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.8364. Min Act: -0.0000

Data Index: 1140920 (The Pile)

Max Activating Token Index: 851

Text #1

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.3428. Min Act: -0.0000

Data Index: 1061643 (The Pile)

Max Activating Token Index: 284

Text #2

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.1717. Min Act: 0.0000

Data Index: 1145452 (The Pile)

Max Activating Token Index: 330

Text #3

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.4213. Min Act: -0.0000

Data Index: 738455 (The Pile)

Max Activating Token Index: 768

Text #4

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.1457. Min Act: -0.0000

Data Index: 505653 (The Pile)

Max Activating Token Index: 960

Text #5

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.1232. Min Act: -0.0000

Data Index: 867892 (The Pile)

Max Activating Token Index: 613

Text #6

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.2298. Min Act: -0.0000

Data Index: 1270341 (The Pile)

Max Activating Token Index: 232

Text #7

Max Range: 3.8364. Min Range: -3.8364

Max Act: 2.9995. Min Act: -0.0000

Data Index: 323056 (The Pile)

Max Activating Token Index: 266

Text #8

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.2630. Min Act: -0.0000

Data Index: 1521079 (The Pile)

Max Activating Token Index: 656

Text #9

Max Range: 3.8364. Min Range: -3.8364

Max Act: 2.9605. Min Act: -0.0000

Data Index: 702426 (The Pile)

Max Activating Token Index: 217

Text #10

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.2684. Min Act: -0.0000

Data Index: 940612 (The Pile)

Max Activating Token Index: 726

Text #11

Max Range: 3.8364. Min Range: -3.8364

Max Act: 2.9462. Min Act: -0.0000

Data Index: 38674 (The Pile)

Max Activating Token Index: 1013

Text #12

Max Range: 3.8364. Min Range: -3.8364

Max Act: 2.9206. Min Act: -0.0000

Data Index: 1549448 (The Pile)

Max Activating Token Index: 234

Text #13

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.0362. Min Act: -0.0000

Data Index: 1239392 (The Pile)

Max Activating Token Index: 432

Text #14

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.3289. Min Act: -0.0000

Data Index: 1897779 (The Pile)

Max Activating Token Index: 680

Text #15

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.0377. Min Act: -0.0000

Data Index: 1230655 (The Pile)

Max Activating Token Index: 585

Text #16

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.1068. Min Act: -0.0000

Data Index: 1729761 (The Pile)

Max Activating Token Index: 106

Text #17

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.0728. Min Act: -0.0000

Data Index: 1064340 (The Pile)

Max Activating Token Index: 1014

Text #18

Max Range: 3.8364. Min Range: -3.8364

Max Act: 3.2144. Min Act: -0.0000

Data Index: 1560118 (The Pile)

Max Activating Token Index: 689

Text #19

Max Range: 3.8364. Min Range: -3.8364

Max Act: 2.6895. Min Act: -0.0000

Data Index: 314562 (The Pile)

Max Activating Token Index: 217