Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2504 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2069. Min Range: -1.2069

Max Act: 1.2069. Min Act: -0.0000

Data Index: 607671 (The Pile)

Max Activating Token Index: 732

Text #1

Max Range: 1.2069. Min Range: -1.2069

Max Act: 1.0417. Min Act: -0.0000

Data Index: 1637965 (The Pile)

Max Activating Token Index: 710

Text #2

Max Range: 1.2069. Min Range: -1.2069

Max Act: 1.0350. Min Act: -0.0000

Data Index: 1739790 (The Pile)

Max Activating Token Index: 945

Text #3

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.9995. Min Act: -0.0000

Data Index: 925432 (The Pile)

Max Activating Token Index: 676

Text #4

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.9279. Min Act: -0.0000

Data Index: 798010 (The Pile)

Max Activating Token Index: 526

Text #5

Max Range: 1.2069. Min Range: -1.2069

Max Act: 1.0050. Min Act: -0.0000

Data Index: 276923 (The Pile)

Max Activating Token Index: 924

Text #6

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8886. Min Act: -0.0000

Data Index: 13979 (The Pile)

Max Activating Token Index: 59

Text #7

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8803. Min Act: -0.0000

Data Index: 1909916 (The Pile)

Max Activating Token Index: 75

Text #8

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8693. Min Act: -0.0000

Data Index: 801134 (The Pile)

Max Activating Token Index: 308

Text #9

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.9914. Min Act: -0.0000

Data Index: 1986289 (The Pile)

Max Activating Token Index: 965

Text #10

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8445. Min Act: -0.0000

Data Index: 1163825 (The Pile)

Max Activating Token Index: 631

Text #11

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8424. Min Act: -0.0000

Data Index: 679609 (The Pile)

Max Activating Token Index: 457

Text #12

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8424. Min Act: -0.0000

Data Index: 1356006 (The Pile)

Max Activating Token Index: 964

Text #13

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8264. Min Act: -0.0000

Data Index: 835884 (The Pile)

Max Activating Token Index: 602

Text #14

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8737. Min Act: -0.0000

Data Index: 988448 (The Pile)

Max Activating Token Index: 544

Text #15

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.7200. Min Act: -0.0000

Data Index: 942735 (The Pile)

Max Activating Token Index: 237

Text #16

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.7557. Min Act: 0.0000

Data Index: 291954 (The Pile)

Max Activating Token Index: 58

Text #17

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.7738. Min Act: -0.0000

Data Index: 675186 (The Pile)

Max Activating Token Index: 105

Text #18

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.8244. Min Act: -0.0000

Data Index: 1269625 (The Pile)

Max Activating Token Index: 206

Text #19

Max Range: 1.2069. Min Range: -1.2069

Max Act: 0.7040. Min Act: -0.0000

Data Index: 1177308 (The Pile)

Max Activating Token Index: 795