Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4924 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8782. Min Act: -0.0000

Data Index: 1568018 (The Pile)

Max Activating Token Index: 547

Text #1

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.9373. Min Act: -0.0000

Data Index: 152463 (The Pile)

Max Activating Token Index: 319

Text #2

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.9130. Min Act: -0.0000

Data Index: 608626 (The Pile)

Max Activating Token Index: 1014

Text #3

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8591. Min Act: -0.0000

Data Index: 137758 (The Pile)

Max Activating Token Index: 564

Text #4

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8064. Min Act: -0.0000

Data Index: 53641 (The Pile)

Max Activating Token Index: 236

Text #5

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8526. Min Act: -0.0000

Data Index: 1953429 (The Pile)

Max Activating Token Index: 452

Text #6

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.6378. Min Act: -0.0000

Data Index: 310730 (The Pile)

Max Activating Token Index: 264

Text #7

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.9171. Min Act: -0.0000

Data Index: 456416 (The Pile)

Max Activating Token Index: 827

Text #8

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.7184. Min Act: -0.0000

Data Index: 1843782 (The Pile)

Max Activating Token Index: 638

Text #9

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.7548. Min Act: -0.0000

Data Index: 1016532 (The Pile)

Max Activating Token Index: 203

Text #10

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8244. Min Act: -0.0000

Data Index: 229268 (The Pile)

Max Activating Token Index: 633

Text #11

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.7271. Min Act: -0.0000

Data Index: 289843 (The Pile)

Max Activating Token Index: 493

Text #12

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.6059. Min Act: -0.0000

Data Index: 1122900 (The Pile)

Max Activating Token Index: 996

Text #13

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.6717. Min Act: -0.0000

Data Index: 1565723 (The Pile)

Max Activating Token Index: 132

Text #14

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.5049. Min Act: -0.0000

Data Index: 242426 (The Pile)

Max Activating Token Index: 1010

Text #15

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.5648. Min Act: -0.0000

Data Index: 231609 (The Pile)

Max Activating Token Index: 592

Text #16

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.6264. Min Act: -0.0000

Data Index: 835006 (The Pile)

Max Activating Token Index: 876

Text #17

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.8004. Min Act: -0.0000

Data Index: 1653635 (The Pile)

Max Activating Token Index: 695

Text #18

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.6172. Min Act: -0.0000

Data Index: 1251022 (The Pile)

Max Activating Token Index: 416

Text #19

Max Range: 1.9373. Min Range: -1.9373

Max Act: 1.5233. Min Act: -0.0000

Data Index: 840563 (The Pile)

Max Activating Token Index: 994