Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3497 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.7609. Min Act: -0.0000

Data Index: 1445450 (The Pile)

Max Activating Token Index: 994

Text #1

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.7713. Min Act: -0.0000

Data Index: 429839 (The Pile)

Max Activating Token Index: 479

Text #2

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.4476. Min Act: -0.0000

Data Index: 1203107 (The Pile)

Max Activating Token Index: 779

Text #3

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.0585. Min Act: -0.0000

Data Index: 1479630 (The Pile)

Max Activating Token Index: 724

Text #4

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.2652. Min Act: -0.0000

Data Index: 1850883 (The Pile)

Max Activating Token Index: 432

Text #5

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.1058. Min Act: -0.0000

Data Index: 936827 (The Pile)

Max Activating Token Index: 81

Text #6

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.9157. Min Act: -0.0000

Data Index: 590783 (The Pile)

Max Activating Token Index: 70

Text #7

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.5255. Min Act: -0.0000

Data Index: 636841 (The Pile)

Max Activating Token Index: 751

Text #8

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.9644. Min Act: -0.0000

Data Index: 1637523 (The Pile)

Max Activating Token Index: 275

Text #9

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.0441. Min Act: -0.0000

Data Index: 1653697 (The Pile)

Max Activating Token Index: 383

Text #10

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.9710. Min Act: -0.0000

Data Index: 188422 (The Pile)

Max Activating Token Index: 919

Text #11

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.0344. Min Act: -0.0000

Data Index: 1237183 (The Pile)

Max Activating Token Index: 603

Text #12

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.8657. Min Act: -0.0000

Data Index: 1877977 (The Pile)

Max Activating Token Index: 671

Text #13

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.1528. Min Act: -0.0000

Data Index: 1949937 (The Pile)

Max Activating Token Index: 151

Text #14

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.0308. Min Act: -0.0000

Data Index: 52010 (The Pile)

Max Activating Token Index: 584

Text #15

Max Range: 3.9710. Min Range: -3.9710

Max Act: 3.1461. Min Act: -0.0000

Data Index: 1463271 (The Pile)

Max Activating Token Index: 933

Text #16

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.7247. Min Act: -0.0000

Data Index: 1775785 (The Pile)

Max Activating Token Index: 723

Text #17

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.7552. Min Act: -0.0000

Data Index: 1229726 (The Pile)

Max Activating Token Index: 40

Text #18

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.7051. Min Act: -0.0000

Data Index: 957183 (The Pile)

Max Activating Token Index: 908

Text #19

Max Range: 3.9710. Min Range: -3.9710

Max Act: 2.4915. Min Act: -0.0000

Data Index: 683674 (The Pile)

Max Activating Token Index: 831