Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3244 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.9987. Min Act: -0.0000

Data Index: 381580 (The Pile)

Max Activating Token Index: 768

Text #1

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.6614. Min Act: -0.0000

Data Index: 1149906 (The Pile)

Max Activating Token Index: 93

Text #2

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.7780. Min Act: -0.0000

Data Index: 1721082 (The Pile)

Max Activating Token Index: 79

Text #3

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.7134. Min Act: -0.0000

Data Index: 1819163 (The Pile)

Max Activating Token Index: 682

Text #4

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4907. Min Act: -0.0000

Data Index: 1970972 (The Pile)

Max Activating Token Index: 252

Text #5

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.5846. Min Act: -0.0000

Data Index: 1801984 (The Pile)

Max Activating Token Index: 655

Text #6

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4106. Min Act: -0.0000

Data Index: 1871629 (The Pile)

Max Activating Token Index: 292

Text #7

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4254. Min Act: -0.0000

Data Index: 431897 (The Pile)

Max Activating Token Index: 370

Text #8

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4208. Min Act: -0.0000

Data Index: 426692 (The Pile)

Max Activating Token Index: 452

Text #9

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.3955. Min Act: -0.0000

Data Index: 814155 (The Pile)

Max Activating Token Index: 680

Text #10

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4027. Min Act: -0.0000

Data Index: 1369203 (The Pile)

Max Activating Token Index: 874

Text #11

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.4211. Min Act: -0.0000

Data Index: 643989 (The Pile)

Max Activating Token Index: 982

Text #12

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.3424. Min Act: -0.0000

Data Index: 1849511 (The Pile)

Max Activating Token Index: 162

Text #13

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.3103. Min Act: -0.0000

Data Index: 1068613 (The Pile)

Max Activating Token Index: 824

Text #14

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.2025. Min Act: -0.0000

Data Index: 1270585 (The Pile)

Max Activating Token Index: 777

Text #15

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.3073. Min Act: -0.0000

Data Index: 1583796 (The Pile)

Max Activating Token Index: 14

Text #16

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.2616. Min Act: -0.0000

Data Index: 1555454 (The Pile)

Max Activating Token Index: 461

Text #17

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.2610. Min Act: -0.0000

Data Index: 1830039 (The Pile)

Max Activating Token Index: 420

Text #18

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.2609. Min Act: -0.0000

Data Index: 1301730 (The Pile)

Max Activating Token Index: 970

Text #19

Max Range: 3.9987. Min Range: -3.9987

Max Act: 3.2454. Min Act: -0.0000

Data Index: 323986 (The Pile)

Max Activating Token Index: 117