Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2628 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5633. Min Act: -0.0000

Data Index: 105978 (The Pile)

Max Activating Token Index: 947

Text #1

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5079. Min Act: -0.0000

Data Index: 1636865 (The Pile)

Max Activating Token Index: 569

Text #2

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.6050. Min Act: -0.0000

Data Index: 1020834 (The Pile)

Max Activating Token Index: 42

Text #3

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5631. Min Act: -0.0000

Data Index: 305406 (The Pile)

Max Activating Token Index: 481

Text #4

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5556. Min Act: -0.0000

Data Index: 907801 (The Pile)

Max Activating Token Index: 791

Text #5

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5690. Min Act: -0.0000

Data Index: 1805394 (The Pile)

Max Activating Token Index: 590

Text #6

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5351. Min Act: -0.0000

Data Index: 1927875 (The Pile)

Max Activating Token Index: 533

Text #7

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5196. Min Act: -0.0000

Data Index: 1763031 (The Pile)

Max Activating Token Index: 620

Text #8

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5315. Min Act: -0.0000

Data Index: 464649 (The Pile)

Max Activating Token Index: 333

Text #9

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5255. Min Act: -0.0000

Data Index: 233437 (The Pile)

Max Activating Token Index: 471

Text #10

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5116. Min Act: -0.0000

Data Index: 1027060 (The Pile)

Max Activating Token Index: 346

Text #11

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.4886. Min Act: -0.0000

Data Index: 196238 (The Pile)

Max Activating Token Index: 160

Text #12

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.4785. Min Act: -0.0000

Data Index: 1470066 (The Pile)

Max Activating Token Index: 964

Text #13

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5013. Min Act: -0.0000

Data Index: 1370495 (The Pile)

Max Activating Token Index: 144

Text #14

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5035. Min Act: -0.0000

Data Index: 1492068 (The Pile)

Max Activating Token Index: 351

Text #15

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5297. Min Act: -0.0000

Data Index: 1412863 (The Pile)

Max Activating Token Index: 548

Text #16

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5217. Min Act: -0.0000

Data Index: 127957 (The Pile)

Max Activating Token Index: 976

Text #17

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.4920. Min Act: -0.0000

Data Index: 483633 (The Pile)

Max Activating Token Index: 311

Text #18

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5080. Min Act: -0.0000

Data Index: 1760400 (The Pile)

Max Activating Token Index: 318

Text #19

Max Range: 0.6050. Min Range: -0.6050

Max Act: 0.5052. Min Act: -0.0000

Data Index: 916003 (The Pile)

Max Activating Token Index: 420