Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1812 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.3229. Min Act: -0.0000

Data Index: 1592716 (The Pile)

Max Activating Token Index: 826

Text #1

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2313. Min Act: -0.0000

Data Index: 1187763 (The Pile)

Max Activating Token Index: 536

Text #2

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2456. Min Act: -0.0000

Data Index: 1849793 (The Pile)

Max Activating Token Index: 1018

Text #3

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2430. Min Act: -0.0000

Data Index: 712347 (The Pile)

Max Activating Token Index: 564

Text #4

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2080. Min Act: -0.0000

Data Index: 1961677 (The Pile)

Max Activating Token Index: 870

Text #5

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2090. Min Act: -0.0000

Data Index: 301704 (The Pile)

Max Activating Token Index: 897

Text #6

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2194. Min Act: -0.0000

Data Index: 1787850 (The Pile)

Max Activating Token Index: 924

Text #7

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1972. Min Act: -0.0000

Data Index: 1623661 (The Pile)

Max Activating Token Index: 590

Text #8

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1876. Min Act: -0.0000

Data Index: 1988960 (The Pile)

Max Activating Token Index: 288

Text #9

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1990. Min Act: -0.0000

Data Index: 1990988 (The Pile)

Max Activating Token Index: 301

Text #10

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1787. Min Act: -0.0000

Data Index: 1637177 (The Pile)

Max Activating Token Index: 804

Text #11

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.2019. Min Act: -0.0000

Data Index: 1445203 (The Pile)

Max Activating Token Index: 649

Text #12

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1758. Min Act: -0.0000

Data Index: 127753 (The Pile)

Max Activating Token Index: 962

Text #13

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1865. Min Act: -0.0000

Data Index: 553135 (The Pile)

Max Activating Token Index: 620

Text #14

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1902. Min Act: -0.0000

Data Index: 1630459 (The Pile)

Max Activating Token Index: 425

Text #15

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1779. Min Act: -0.0000

Data Index: 1673980 (The Pile)

Max Activating Token Index: 886

Text #16

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1875. Min Act: -0.0000

Data Index: 1949911 (The Pile)

Max Activating Token Index: 290

Text #17

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1915. Min Act: -0.0000

Data Index: 1074976 (The Pile)

Max Activating Token Index: 670

Text #18

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1784. Min Act: -0.0000

Data Index: 343767 (The Pile)

Max Activating Token Index: 945

Text #19

Max Range: 0.3229. Min Range: -0.3229

Max Act: 0.1780. Min Act: -0.0000

Data Index: 1095966 (The Pile)

Max Activating Token Index: 327