Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 79 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2291. Min Act: -0.0000

Data Index: 1460982 (The Pile)

Max Activating Token Index: 398

Text #1

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2248. Min Act: -0.0000

Data Index: 1990508 (The Pile)

Max Activating Token Index: 682

Text #2

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2275. Min Act: -0.0000

Data Index: 1443395 (The Pile)

Max Activating Token Index: 959

Text #3

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2150. Min Act: -0.0000

Data Index: 749349 (The Pile)

Max Activating Token Index: 585

Text #4

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2155. Min Act: -0.0000

Data Index: 1967520 (The Pile)

Max Activating Token Index: 618

Text #5

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2129. Min Act: -0.0000

Data Index: 1985611 (The Pile)

Max Activating Token Index: 705

Text #6

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2049. Min Act: -0.0000

Data Index: 1120817 (The Pile)

Max Activating Token Index: 186

Text #7

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2076. Min Act: -0.0000

Data Index: 1247474 (The Pile)

Max Activating Token Index: 923

Text #8

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2172. Min Act: -0.0000

Data Index: 736350 (The Pile)

Max Activating Token Index: 587

Text #9

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2177. Min Act: -0.0000

Data Index: 1875696 (The Pile)

Max Activating Token Index: 400

Text #10

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2101. Min Act: -0.0000

Data Index: 587834 (The Pile)

Max Activating Token Index: 97

Text #11

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2099. Min Act: -0.0000

Data Index: 589021 (The Pile)

Max Activating Token Index: 561

Text #12

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2163. Min Act: -0.0000

Data Index: 641344 (The Pile)

Max Activating Token Index: 48

Text #13

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2086. Min Act: -0.0000

Data Index: 1275679 (The Pile)

Max Activating Token Index: 517

Text #14

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2009. Min Act: -0.0000

Data Index: 556710 (The Pile)

Max Activating Token Index: 170

Text #15

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2017. Min Act: -0.0000

Data Index: 791756 (The Pile)

Max Activating Token Index: 563

Text #16

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2143. Min Act: -0.0000

Data Index: 345254 (The Pile)

Max Activating Token Index: 194

Text #17

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.1965. Min Act: -0.0000

Data Index: 1264963 (The Pile)

Max Activating Token Index: 898

Text #18

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2065. Min Act: -0.0000

Data Index: 1868442 (The Pile)

Max Activating Token Index: 868

Text #19

Max Range: 0.2291. Min Range: -0.2291

Max Act: 0.2141. Min Act: -0.0000

Data Index: 1133916 (The Pile)

Max Activating Token Index: 92