Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3658 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1685. Min Act: -0.0000

Data Index: 1900982 (The Pile)

Max Activating Token Index: 601

Text #1

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1546. Min Act: -0.0000

Data Index: 1227212 (The Pile)

Max Activating Token Index: 460

Text #2

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1411. Min Act: -0.0000

Data Index: 1280289 (The Pile)

Max Activating Token Index: 753

Text #3

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1369. Min Act: -0.0000

Data Index: 225446 (The Pile)

Max Activating Token Index: 857

Text #4

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1236. Min Act: -0.0000

Data Index: 159531 (The Pile)

Max Activating Token Index: 883

Text #5

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1268. Min Act: -0.0000

Data Index: 1390839 (The Pile)

Max Activating Token Index: 250

Text #6

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1110. Min Act: -0.0000

Data Index: 1830148 (The Pile)

Max Activating Token Index: 984

Text #7

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1154. Min Act: -0.0000

Data Index: 160327 (The Pile)

Max Activating Token Index: 1007

Text #8

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1247. Min Act: -0.0000

Data Index: 371011 (The Pile)

Max Activating Token Index: 601

Text #9

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1107. Min Act: -0.0000

Data Index: 716823 (The Pile)

Max Activating Token Index: 827

Text #10

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1130. Min Act: -0.0000

Data Index: 1245561 (The Pile)

Max Activating Token Index: 114

Text #11

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1114. Min Act: -0.0000

Data Index: 448938 (The Pile)

Max Activating Token Index: 308

Text #12

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1147. Min Act: -0.0000

Data Index: 1391120 (The Pile)

Max Activating Token Index: 844

Text #13

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1150. Min Act: -0.0000

Data Index: 916453 (The Pile)

Max Activating Token Index: 162

Text #14

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1058. Min Act: -0.0000

Data Index: 107354 (The Pile)

Max Activating Token Index: 224

Text #15

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1092. Min Act: -0.0000

Data Index: 932710 (The Pile)

Max Activating Token Index: 665

Text #16

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1158. Min Act: -0.0000

Data Index: 1871122 (The Pile)

Max Activating Token Index: 945

Text #17

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1062. Min Act: -0.0000

Data Index: 1991402 (The Pile)

Max Activating Token Index: 429

Text #18

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1058. Min Act: -0.0000

Data Index: 324057 (The Pile)

Max Activating Token Index: 596

Text #19

Max Range: 0.1685. Min Range: -0.1685

Max Act: 0.1021. Min Act: -0.0000

Data Index: 656015 (The Pile)

Max Activating Token Index: 84