Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2031 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.0640. Min Range: -3.0640

Max Act: 3.0640. Min Act: -0.0000

Data Index: 1563946 (The Pile)

Max Activating Token Index: 964

Text #1

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.7303. Min Act: -0.0000

Data Index: 1358053 (The Pile)

Max Activating Token Index: 629

Text #2

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.5454. Min Act: -0.0000

Data Index: 297561 (The Pile)

Max Activating Token Index: 749

Text #3

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.7306. Min Act: -0.0000

Data Index: 1495515 (The Pile)

Max Activating Token Index: 641

Text #4

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.4337. Min Act: -0.0000

Data Index: 892437 (The Pile)

Max Activating Token Index: 805

Text #5

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3742. Min Act: -0.0000

Data Index: 170346 (The Pile)

Max Activating Token Index: 352

Text #6

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3614. Min Act: -0.0000

Data Index: 1575186 (The Pile)

Max Activating Token Index: 750

Text #7

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.4859. Min Act: -0.0000

Data Index: 1872851 (The Pile)

Max Activating Token Index: 855

Text #8

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3542. Min Act: -0.0000

Data Index: 441880 (The Pile)

Max Activating Token Index: 1019

Text #9

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.2341. Min Act: -0.0000

Data Index: 1369968 (The Pile)

Max Activating Token Index: 742

Text #10

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3485. Min Act: -0.0000

Data Index: 1289660 (The Pile)

Max Activating Token Index: 494

Text #11

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3209. Min Act: -0.0000

Data Index: 1247491 (The Pile)

Max Activating Token Index: 273

Text #12

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.5421. Min Act: -0.0000

Data Index: 1065532 (The Pile)

Max Activating Token Index: 685

Text #13

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3910. Min Act: -0.0000

Data Index: 1881899 (The Pile)

Max Activating Token Index: 876

Text #14

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3710. Min Act: -0.0000

Data Index: 1080906 (The Pile)

Max Activating Token Index: 523

Text #15

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.4000. Min Act: -0.0000

Data Index: 1658162 (The Pile)

Max Activating Token Index: 1015

Text #16

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.2651. Min Act: -0.0000

Data Index: 1131256 (The Pile)

Max Activating Token Index: 836

Text #17

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.3902. Min Act: -0.0000

Data Index: 1962752 (The Pile)

Max Activating Token Index: 402

Text #18

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.6210. Min Act: -0.0000

Data Index: 1949950 (The Pile)

Max Activating Token Index: 800

Text #19

Max Range: 3.0640. Min Range: -3.0640

Max Act: 2.2642. Min Act: -0.0000

Data Index: 1138157 (The Pile)

Max Activating Token Index: 642