Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3186 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.5900. Min Act: -0.0000

Data Index: 1023684 (The Pile)

Max Activating Token Index: 715

Text #1

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.5094. Min Act: -0.0000

Data Index: 512236 (The Pile)

Max Activating Token Index: 452

Text #2

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.5645. Min Act: -0.0000

Data Index: 312075 (The Pile)

Max Activating Token Index: 596

Text #3

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.4168. Min Act: -0.0000

Data Index: 1120413 (The Pile)

Max Activating Token Index: 628

Text #4

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.5471. Min Act: -0.0000

Data Index: 954128 (The Pile)

Max Activating Token Index: 890

Text #5

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.3107. Min Act: -0.0000

Data Index: 386979 (The Pile)

Max Activating Token Index: 251

Text #6

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.4178. Min Act: -0.0000

Data Index: 99642 (The Pile)

Max Activating Token Index: 494

Text #7

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.3349. Min Act: -0.0000

Data Index: 1246721 (The Pile)

Max Activating Token Index: 453

Text #8

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.1542. Min Act: -0.0000

Data Index: 1907053 (The Pile)

Max Activating Token Index: 925

Text #9

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.6211. Min Act: -0.0000

Data Index: 957425 (The Pile)

Max Activating Token Index: 843

Text #10

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.3128. Min Act: -0.0000

Data Index: 702366 (The Pile)

Max Activating Token Index: 783

Text #11

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.2497. Min Act: -0.0000

Data Index: 909925 (The Pile)

Max Activating Token Index: 537

Text #12

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.2329. Min Act: -0.0000

Data Index: 362694 (The Pile)

Max Activating Token Index: 171

Text #13

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.3342. Min Act: -0.0000

Data Index: 1261738 (The Pile)

Max Activating Token Index: 443

Text #14

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.2296. Min Act: -0.0000

Data Index: 1774234 (The Pile)

Max Activating Token Index: 235

Text #15

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.1711. Min Act: -0.0000

Data Index: 672942 (The Pile)

Max Activating Token Index: 314

Text #16

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.3837. Min Act: -0.0000

Data Index: 1353742 (The Pile)

Max Activating Token Index: 928

Text #17

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.2371. Min Act: -0.0000

Data Index: 235366 (The Pile)

Max Activating Token Index: 232

Text #18

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.1941. Min Act: -0.0000

Data Index: 276672 (The Pile)

Max Activating Token Index: 172

Text #19

Max Range: 1.6211. Min Range: -1.6211

Max Act: 1.2710. Min Act: -0.0000

Data Index: 1830710 (The Pile)

Max Activating Token Index: 566