Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1092 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2627. Min Range: -1.2627

Max Act: 1.2627. Min Act: -0.0000

Data Index: 358882 (The Pile)

Max Activating Token Index: 68

Text #1

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.9687. Min Act: -0.0000

Data Index: 614941 (The Pile)

Max Activating Token Index: 82

Text #2

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.9086. Min Act: -0.0000

Data Index: 714340 (The Pile)

Max Activating Token Index: 806

Text #3

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.9482. Min Act: -0.0000

Data Index: 1355737 (The Pile)

Max Activating Token Index: 100

Text #4

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.8127. Min Act: -0.0000

Data Index: 1109718 (The Pile)

Max Activating Token Index: 1015

Text #5

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.8212. Min Act: -0.0000

Data Index: 677725 (The Pile)

Max Activating Token Index: 832

Text #6

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.8582. Min Act: -0.0000

Data Index: 681555 (The Pile)

Max Activating Token Index: 87

Text #7

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.8144. Min Act: -0.0000

Data Index: 772891 (The Pile)

Max Activating Token Index: 111

Text #8

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7941. Min Act: -0.0000

Data Index: 572326 (The Pile)

Max Activating Token Index: 356

Text #9

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.6840. Min Act: -0.0000

Data Index: 1081721 (The Pile)

Max Activating Token Index: 818

Text #10

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7617. Min Act: -0.0000

Data Index: 1782571 (The Pile)

Max Activating Token Index: 386

Text #11

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7809. Min Act: -0.0000

Data Index: 1970542 (The Pile)

Max Activating Token Index: 579

Text #12

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7411. Min Act: -0.0000

Data Index: 1520254 (The Pile)

Max Activating Token Index: 353

Text #13

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7069. Min Act: -0.0000

Data Index: 720191 (The Pile)

Max Activating Token Index: 653

Text #14

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7589. Min Act: -0.0000

Data Index: 1528430 (The Pile)

Max Activating Token Index: 975

Text #15

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7477. Min Act: -0.0000

Data Index: 183411 (The Pile)

Max Activating Token Index: 463

Text #16

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.6360. Min Act: -0.0000

Data Index: 1464697 (The Pile)

Max Activating Token Index: 1002

Text #17

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.6981. Min Act: -0.0000

Data Index: 726076 (The Pile)

Max Activating Token Index: 557

Text #18

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.7493. Min Act: -0.0000

Data Index: 99876 (The Pile)

Max Activating Token Index: 770

Text #19

Max Range: 1.2627. Min Range: -1.2627

Max Act: 0.6633. Min Act: -0.0000

Data Index: 499037 (The Pile)

Max Activating Token Index: 203