Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1371 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.2302. Min Range: -5.2302

Max Act: 5.2302. Min Act: -0.0000

Data Index: 1004844 (The Pile)

Max Activating Token Index: 73

Text #1

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.5215. Min Act: -0.0000

Data Index: 1499970 (The Pile)

Max Activating Token Index: 879

Text #2

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.0515. Min Act: -0.0000

Data Index: 1134301 (The Pile)

Max Activating Token Index: 279

Text #3

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.8702. Min Act: -0.0000

Data Index: 1963494 (The Pile)

Max Activating Token Index: 952

Text #4

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.0088. Min Act: -0.0000

Data Index: 657760 (The Pile)

Max Activating Token Index: 176

Text #5

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.1203. Min Act: -0.0000

Data Index: 1132289 (The Pile)

Max Activating Token Index: 81

Text #6

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.8237. Min Act: -0.0000

Data Index: 1452165 (The Pile)

Max Activating Token Index: 412

Text #7

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.1069. Min Act: -0.0000

Data Index: 1870824 (The Pile)

Max Activating Token Index: 132

Text #8

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.7267. Min Act: -0.0000

Data Index: 683729 (The Pile)

Max Activating Token Index: 968

Text #9

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.0021. Min Act: -0.0000

Data Index: 763141 (The Pile)

Max Activating Token Index: 187

Text #10

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.1703. Min Act: -0.0000

Data Index: 47408 (The Pile)

Max Activating Token Index: 806

Text #11

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.8218. Min Act: -0.0000

Data Index: 599047 (The Pile)

Max Activating Token Index: 473

Text #12

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.9853. Min Act: -0.0000

Data Index: 456080 (The Pile)

Max Activating Token Index: 411

Text #13

Max Range: 5.2302. Min Range: -5.2302

Max Act: 4.1622. Min Act: -0.0000

Data Index: 836489 (The Pile)

Max Activating Token Index: 359

Text #14

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.9773. Min Act: -0.0000

Data Index: 1452716 (The Pile)

Max Activating Token Index: 749

Text #15

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.6750. Min Act: -0.0000

Data Index: 765957 (The Pile)

Max Activating Token Index: 509

Text #16

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.9695. Min Act: -0.0000

Data Index: 68503 (The Pile)

Max Activating Token Index: 88

Text #17

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.9453. Min Act: -0.0000

Data Index: 1224349 (The Pile)

Max Activating Token Index: 565

Text #18

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.8844. Min Act: -0.0000

Data Index: 1483151 (The Pile)

Max Activating Token Index: 751

Text #19

Max Range: 5.2302. Min Range: -5.2302

Max Act: 3.9605. Min Act: -0.0000

Data Index: 139829 (The Pile)

Max Activating Token Index: 439