Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 523 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.2222. Min Act: -0.0000

Data Index: 59118 (The Pile)

Max Activating Token Index: 79

Text #1

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.1972. Min Act: -0.0000

Data Index: 1165177 (The Pile)

Max Activating Token Index: 128

Text #2

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.1216. Min Act: -0.0000

Data Index: 224547 (The Pile)

Max Activating Token Index: 115

Text #3

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.0132. Min Act: -0.0000

Data Index: 1179279 (The Pile)

Max Activating Token Index: 603

Text #4

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.0072. Min Act: -0.0000

Data Index: 1670305 (The Pile)

Max Activating Token Index: 563

Text #5

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9553. Min Act: -0.0000

Data Index: 1195564 (The Pile)

Max Activating Token Index: 254

Text #6

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9722. Min Act: -0.0000

Data Index: 182541 (The Pile)

Max Activating Token Index: 620

Text #7

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9652. Min Act: -0.0000

Data Index: 447720 (The Pile)

Max Activating Token Index: 100

Text #8

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9201. Min Act: -0.0000

Data Index: 1058366 (The Pile)

Max Activating Token Index: 363

Text #9

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9784. Min Act: -0.0000

Data Index: 1099104 (The Pile)

Max Activating Token Index: 521

Text #10

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9637. Min Act: -0.0000

Data Index: 135100 (The Pile)

Max Activating Token Index: 336

Text #11

Max Range: 1.2222. Min Range: -1.2222

Max Act: 1.0224. Min Act: -0.0000

Data Index: 1060915 (The Pile)

Max Activating Token Index: 589

Text #12

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.8572. Min Act: -0.0000

Data Index: 294740 (The Pile)

Max Activating Token Index: 262

Text #13

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9085. Min Act: -0.0000

Data Index: 1885352 (The Pile)

Max Activating Token Index: 408

Text #14

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9592. Min Act: -0.0000

Data Index: 1915456 (The Pile)

Max Activating Token Index: 904

Text #15

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.8746. Min Act: -0.0000

Data Index: 697655 (The Pile)

Max Activating Token Index: 154

Text #16

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.8987. Min Act: -0.0000

Data Index: 129652 (The Pile)

Max Activating Token Index: 227

Text #17

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.9269. Min Act: -0.0000

Data Index: 796124 (The Pile)

Max Activating Token Index: 984

Text #18

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.8990. Min Act: -0.0000

Data Index: 1355641 (The Pile)

Max Activating Token Index: 388

Text #19

Max Range: 1.2222. Min Range: -1.2222

Max Act: 0.8648. Min Act: -0.0000

Data Index: 1076515 (The Pile)

Max Activating Token Index: 135