Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3995 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.8206. Min Act: -0.0000

Data Index: 1068709 (The Pile)

Max Activating Token Index: 209

Text #1

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.7237. Min Act: -0.0000

Data Index: 551668 (The Pile)

Max Activating Token Index: 135

Text #2

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.7714. Min Act: -0.0000

Data Index: 1415864 (The Pile)

Max Activating Token Index: 738

Text #3

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.6839. Min Act: -0.0000

Data Index: 300254 (The Pile)

Max Activating Token Index: 773

Text #4

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.5547. Min Act: -0.0000

Data Index: 474894 (The Pile)

Max Activating Token Index: 538

Text #5

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.4295. Min Act: -0.0000

Data Index: 1518700 (The Pile)

Max Activating Token Index: 890

Text #6

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.4443. Min Act: -0.0000

Data Index: 232133 (The Pile)

Max Activating Token Index: 264

Text #7

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.6625. Min Act: -0.0000

Data Index: 1413224 (The Pile)

Max Activating Token Index: 249

Text #8

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2738. Min Act: -0.0000

Data Index: 197973 (The Pile)

Max Activating Token Index: 137

Text #9

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.3052. Min Act: -0.0000

Data Index: 493643 (The Pile)

Max Activating Token Index: 76

Text #10

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.4804. Min Act: -0.0000

Data Index: 1435133 (The Pile)

Max Activating Token Index: 207

Text #11

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.5154. Min Act: -0.0000

Data Index: 34673 (The Pile)

Max Activating Token Index: 78

Text #12

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.3975. Min Act: -0.0000

Data Index: 1097683 (The Pile)

Max Activating Token Index: 937

Text #13

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.3556. Min Act: -0.0000

Data Index: 473090 (The Pile)

Max Activating Token Index: 127

Text #14

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2915. Min Act: -0.0000

Data Index: 257391 (The Pile)

Max Activating Token Index: 14

Text #15

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2556. Min Act: -0.0000

Data Index: 1094027 (The Pile)

Max Activating Token Index: 89

Text #16

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2593. Min Act: -0.0000

Data Index: 648545 (The Pile)

Max Activating Token Index: 283

Text #17

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2036. Min Act: -0.0000

Data Index: 1902981 (The Pile)

Max Activating Token Index: 597

Text #18

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.2391. Min Act: -0.0000

Data Index: 428386 (The Pile)

Max Activating Token Index: 762

Text #19

Max Range: 1.8206. Min Range: -1.8206

Max Act: 1.4355. Min Act: -0.0000

Data Index: 1125241 (The Pile)

Max Activating Token Index: 26