Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2904 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.9544. Min Act: -0.0000

Data Index: 154076 (The Pile)

Max Activating Token Index: 492

Text #1

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.8607. Min Act: -0.0000

Data Index: 242953 (The Pile)

Max Activating Token Index: 151

Text #2

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7902. Min Act: -0.0000

Data Index: 1288309 (The Pile)

Max Activating Token Index: 171

Text #3

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.8035. Min Act: -0.0000

Data Index: 645968 (The Pile)

Max Activating Token Index: 475

Text #4

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7021. Min Act: -0.0000

Data Index: 322269 (The Pile)

Max Activating Token Index: 723

Text #5

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7875. Min Act: -0.0000

Data Index: 1495194 (The Pile)

Max Activating Token Index: 266

Text #6

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7759. Min Act: -0.0000

Data Index: 376739 (The Pile)

Max Activating Token Index: 562

Text #7

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7490. Min Act: -0.0000

Data Index: 1032810 (The Pile)

Max Activating Token Index: 380

Text #8

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7032. Min Act: -0.0000

Data Index: 14778 (The Pile)

Max Activating Token Index: 857

Text #9

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7211. Min Act: -0.0000

Data Index: 67914 (The Pile)

Max Activating Token Index: 249

Text #10

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7187. Min Act: -0.0000

Data Index: 407659 (The Pile)

Max Activating Token Index: 628

Text #11

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7270. Min Act: -0.0000

Data Index: 1278170 (The Pile)

Max Activating Token Index: 326

Text #12

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7333. Min Act: -0.0000

Data Index: 438890 (The Pile)

Max Activating Token Index: 463

Text #13

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.6866. Min Act: -0.0000

Data Index: 73425 (The Pile)

Max Activating Token Index: 621

Text #14

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.6836. Min Act: -0.0000

Data Index: 959335 (The Pile)

Max Activating Token Index: 764

Text #15

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7081. Min Act: -0.0000

Data Index: 778324 (The Pile)

Max Activating Token Index: 976

Text #16

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7261. Min Act: -0.0000

Data Index: 1689627 (The Pile)

Max Activating Token Index: 688

Text #17

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.6976. Min Act: -0.0000

Data Index: 1323100 (The Pile)

Max Activating Token Index: 457

Text #18

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.7039. Min Act: -0.0000

Data Index: 1579864 (The Pile)

Max Activating Token Index: 381

Text #19

Max Range: 0.9544. Min Range: -0.9544

Max Act: 0.6521. Min Act: -0.0000

Data Index: 417034 (The Pile)

Max Activating Token Index: 231