Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2059 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.1092. Min Range: -5.1092

Max Act: 5.0019. Min Act: -0.0000

Data Index: 282171 (The Pile)

Max Activating Token Index: 112

Text #1

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.5821. Min Act: -0.0000

Data Index: 1676326 (The Pile)

Max Activating Token Index: 751

Text #2

Max Range: 5.1092. Min Range: -5.1092

Max Act: 5.1092. Min Act: -0.0000

Data Index: 617328 (The Pile)

Max Activating Token Index: 451

Text #3

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.9170. Min Act: -0.0000

Data Index: 1908129 (The Pile)

Max Activating Token Index: 639

Text #4

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.7367. Min Act: -0.0000

Data Index: 1548923 (The Pile)

Max Activating Token Index: 1008

Text #5

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.6746. Min Act: -0.0000

Data Index: 539418 (The Pile)

Max Activating Token Index: 991

Text #6

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.8096. Min Act: -0.0000

Data Index: 1543554 (The Pile)

Max Activating Token Index: 176

Text #7

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.5894. Min Act: -0.0000

Data Index: 1890278 (The Pile)

Max Activating Token Index: 703

Text #8

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.9599. Min Act: -0.0000

Data Index: 1457477 (The Pile)

Max Activating Token Index: 81

Text #9

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.2579. Min Act: -0.0000

Data Index: 1737126 (The Pile)

Max Activating Token Index: 909

Text #10

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.7560. Min Act: -0.0000

Data Index: 1530186 (The Pile)

Max Activating Token Index: 179

Text #11

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.7560. Min Act: -0.0000

Data Index: 1087600 (The Pile)

Max Activating Token Index: 985

Text #12

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.7077. Min Act: -0.0000

Data Index: 1090999 (The Pile)

Max Activating Token Index: 830

Text #13

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.7313. Min Act: -0.0000

Data Index: 1141165 (The Pile)

Max Activating Token Index: 412

Text #14

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.9119. Min Act: -0.0000

Data Index: 1722919 (The Pile)

Max Activating Token Index: 600

Text #15

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.6855. Min Act: -0.0000

Data Index: 804642 (The Pile)

Max Activating Token Index: 667

Text #16

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.5124. Min Act: -0.0000

Data Index: 622955 (The Pile)

Max Activating Token Index: 149

Text #17

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.5039. Min Act: -0.0000

Data Index: 206051 (The Pile)

Max Activating Token Index: 71

Text #18

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.5035. Min Act: -0.0000

Data Index: 634041 (The Pile)

Max Activating Token Index: 99

Text #19

Max Range: 5.1092. Min Range: -5.1092

Max Act: 4.6467. Min Act: -0.0000

Data Index: 1374728 (The Pile)

Max Activating Token Index: 362