Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2747 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.0945. Min Range: -4.0945

Max Act: 4.0945. Min Act: -0.0000

Data Index: 1093440 (The Pile)

Max Activating Token Index: 275

Text #1

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2612. Min Act: -0.0000

Data Index: 1637544 (The Pile)

Max Activating Token Index: 844

Text #2

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.1711. Min Act: -0.0000

Data Index: 333318 (The Pile)

Max Activating Token Index: 440

Text #3

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2372. Min Act: -0.0000

Data Index: 1033649 (The Pile)

Max Activating Token Index: 87

Text #4

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.1384. Min Act: -0.0000

Data Index: 748021 (The Pile)

Max Activating Token Index: 250

Text #5

Max Range: 4.0945. Min Range: -4.0945

Max Act: 2.9416. Min Act: -0.0000

Data Index: 331325 (The Pile)

Max Activating Token Index: 1016

Text #6

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2243. Min Act: -0.0000

Data Index: 747290 (The Pile)

Max Activating Token Index: 414

Text #7

Max Range: 4.0945. Min Range: -4.0945

Max Act: 2.9060. Min Act: -0.0000

Data Index: 236839 (The Pile)

Max Activating Token Index: 1014

Text #8

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3614. Min Act: -0.0000

Data Index: 1901904 (The Pile)

Max Activating Token Index: 956

Text #9

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2244. Min Act: -0.0000

Data Index: 1321558 (The Pile)

Max Activating Token Index: 58

Text #10

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2222. Min Act: -0.0000

Data Index: 1082893 (The Pile)

Max Activating Token Index: 489

Text #11

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.0891. Min Act: -0.0000

Data Index: 617366 (The Pile)

Max Activating Token Index: 999

Text #12

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.0620. Min Act: -0.0000

Data Index: 548843 (The Pile)

Max Activating Token Index: 271

Text #13

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.2185. Min Act: -0.0000

Data Index: 1834122 (The Pile)

Max Activating Token Index: 392

Text #14

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3367. Min Act: -0.0000

Data Index: 225389 (The Pile)

Max Activating Token Index: 382

Text #15

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.0546. Min Act: -0.0000

Data Index: 1703198 (The Pile)

Max Activating Token Index: 180

Text #16

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3544. Min Act: -0.0000

Data Index: 1584292 (The Pile)

Max Activating Token Index: 470

Text #17

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3542. Min Act: -0.0000

Data Index: 1209656 (The Pile)

Max Activating Token Index: 732

Text #18

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3393. Min Act: -0.0000

Data Index: 1726226 (The Pile)

Max Activating Token Index: 459

Text #19

Max Range: 4.0945. Min Range: -4.0945

Max Act: 3.3370. Min Act: -0.0000

Data Index: 144462 (The Pile)

Max Activating Token Index: 579