Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3860 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.6086. Min Act: -0.0000

Data Index: 787334 (The Pile)

Max Activating Token Index: 249

Text #1

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.7444. Min Act: -0.0000

Data Index: 1240745 (The Pile)

Max Activating Token Index: 247

Text #2

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.6020. Min Act: -0.0000

Data Index: 1061422 (The Pile)

Max Activating Token Index: 323

Text #3

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.6247. Min Act: -0.0000

Data Index: 1637023 (The Pile)

Max Activating Token Index: 693

Text #4

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.6724. Min Act: -0.0000

Data Index: 1759327 (The Pile)

Max Activating Token Index: 718

Text #5

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.6114. Min Act: -0.0000

Data Index: 1937485 (The Pile)

Max Activating Token Index: 890

Text #6

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5152. Min Act: -0.0000

Data Index: 193726 (The Pile)

Max Activating Token Index: 384

Text #7

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5737. Min Act: -0.0000

Data Index: 1362475 (The Pile)

Max Activating Token Index: 613

Text #8

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5024. Min Act: -0.0000

Data Index: 953277 (The Pile)

Max Activating Token Index: 546

Text #9

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5900. Min Act: -0.0000

Data Index: 1595643 (The Pile)

Max Activating Token Index: 548

Text #10

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5679. Min Act: -0.0000

Data Index: 1246466 (The Pile)

Max Activating Token Index: 609

Text #11

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5246. Min Act: -0.0000

Data Index: 1250538 (The Pile)

Max Activating Token Index: 949

Text #12

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5125. Min Act: -0.0000

Data Index: 1308201 (The Pile)

Max Activating Token Index: 213

Text #13

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.4029. Min Act: -0.0000

Data Index: 851999 (The Pile)

Max Activating Token Index: 926

Text #14

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.4852. Min Act: -0.0000

Data Index: 980882 (The Pile)

Max Activating Token Index: 264

Text #15

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.4763. Min Act: -0.0000

Data Index: 1684816 (The Pile)

Max Activating Token Index: 1017

Text #16

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5214. Min Act: -0.0000

Data Index: 1119882 (The Pile)

Max Activating Token Index: 385

Text #17

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.4283. Min Act: -0.0000

Data Index: 1871843 (The Pile)

Max Activating Token Index: 255

Text #18

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.4410. Min Act: -0.0000

Data Index: 303445 (The Pile)

Max Activating Token Index: 530

Text #19

Max Range: 0.7444. Min Range: -0.7444

Max Act: 0.5012. Min Act: -0.0000

Data Index: 108495 (The Pile)

Max Activating Token Index: 222