Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3981 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.7597. Min Act: -0.0000

Data Index: 1765169 (The Pile)

Max Activating Token Index: 351

Text #1

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.7039. Min Act: -0.0000

Data Index: 530738 (The Pile)

Max Activating Token Index: 856

Text #2

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6630. Min Act: -0.0000

Data Index: 1093347 (The Pile)

Max Activating Token Index: 95

Text #3

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6518. Min Act: -0.0000

Data Index: 454210 (The Pile)

Max Activating Token Index: 334

Text #4

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6382. Min Act: -0.0000

Data Index: 373994 (The Pile)

Max Activating Token Index: 441

Text #5

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5993. Min Act: -0.0000

Data Index: 1916490 (The Pile)

Max Activating Token Index: 392

Text #6

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.6124. Min Act: -0.0000

Data Index: 1783289 (The Pile)

Max Activating Token Index: 185

Text #7

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5965. Min Act: -0.0000

Data Index: 143832 (The Pile)

Max Activating Token Index: 158

Text #8

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5751. Min Act: -0.0000

Data Index: 845967 (The Pile)

Max Activating Token Index: 297

Text #9

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5494. Min Act: -0.0000

Data Index: 1964253 (The Pile)

Max Activating Token Index: 710

Text #10

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5722. Min Act: -0.0000

Data Index: 1506169 (The Pile)

Max Activating Token Index: 1017

Text #11

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5518. Min Act: -0.0000

Data Index: 879425 (The Pile)

Max Activating Token Index: 961

Text #12

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5336. Min Act: -0.0000

Data Index: 1146485 (The Pile)

Max Activating Token Index: 113

Text #13

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5509. Min Act: -0.0000

Data Index: 681991 (The Pile)

Max Activating Token Index: 510

Text #14

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5281. Min Act: -0.0000

Data Index: 1437792 (The Pile)

Max Activating Token Index: 705

Text #15

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5194. Min Act: -0.0000

Data Index: 1799634 (The Pile)

Max Activating Token Index: 19

Text #16

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5551. Min Act: -0.0000

Data Index: 264157 (The Pile)

Max Activating Token Index: 695

Text #17

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5455. Min Act: -0.0000

Data Index: 104067 (The Pile)

Max Activating Token Index: 218

Text #18

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5374. Min Act: -0.0000

Data Index: 1841663 (The Pile)

Max Activating Token Index: 760

Text #19

Max Range: 0.7597. Min Range: -0.7597

Max Act: 0.5225. Min Act: -0.0000

Data Index: 502597 (The Pile)

Max Activating Token Index: 299