Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1673 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.2595. Min Act: -0.0000

Data Index: 22998 (The Pile)

Max Activating Token Index: 342

Text #1

Max Range: 4.0646. Min Range: -4.0646

Max Act: 4.0646. Min Act: -0.0000

Data Index: 468422 (The Pile)

Max Activating Token Index: 110

Text #2

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.4746. Min Act: -0.0000

Data Index: 102242 (The Pile)

Max Activating Token Index: 324

Text #3

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.3268. Min Act: -0.0000

Data Index: 1432864 (The Pile)

Max Activating Token Index: 396

Text #4

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.3337. Min Act: -0.0000

Data Index: 1438796 (The Pile)

Max Activating Token Index: 963

Text #5

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.1735. Min Act: -0.0000

Data Index: 1612562 (The Pile)

Max Activating Token Index: 461

Text #6

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.4601. Min Act: -0.0000

Data Index: 182070 (The Pile)

Max Activating Token Index: 339

Text #7

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.4780. Min Act: -0.0000

Data Index: 308232 (The Pile)

Max Activating Token Index: 329

Text #8

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.6477. Min Act: -0.0000

Data Index: 512532 (The Pile)

Max Activating Token Index: 122

Text #9

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.4540. Min Act: -0.0000

Data Index: 1714616 (The Pile)

Max Activating Token Index: 298

Text #10

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.2138. Min Act: -0.0000

Data Index: 802215 (The Pile)

Max Activating Token Index: 802

Text #11

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.1927. Min Act: -0.0000

Data Index: 1056073 (The Pile)

Max Activating Token Index: 207

Text #12

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.1751. Min Act: -0.0000

Data Index: 7249 (The Pile)

Max Activating Token Index: 402

Text #13

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.6600. Min Act: -0.0000

Data Index: 462046 (The Pile)

Max Activating Token Index: 171

Text #14

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.3317. Min Act: -0.0000

Data Index: 151927 (The Pile)

Max Activating Token Index: 491

Text #15

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.1724. Min Act: -0.0000

Data Index: 1678530 (The Pile)

Max Activating Token Index: 507

Text #16

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.3486. Min Act: -0.0000

Data Index: 1723028 (The Pile)

Max Activating Token Index: 280

Text #17

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.1858. Min Act: -0.0000

Data Index: 126867 (The Pile)

Max Activating Token Index: 479

Text #18

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.4427. Min Act: -0.0000

Data Index: 145900 (The Pile)

Max Activating Token Index: 407

Text #19

Max Range: 4.0646. Min Range: -4.0646

Max Act: 3.3213. Min Act: -0.0000

Data Index: 366705 (The Pile)

Max Activating Token Index: 333