Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3122 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8954. Min Act: -0.0000

Data Index: 1602219 (The Pile)

Max Activating Token Index: 186

Text #1

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8915. Min Act: -0.0000

Data Index: 1760868 (The Pile)

Max Activating Token Index: 604

Text #2

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8890. Min Act: -0.0000

Data Index: 888242 (The Pile)

Max Activating Token Index: 296

Text #3

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.9863. Min Act: -0.0000

Data Index: 1300705 (The Pile)

Max Activating Token Index: 788

Text #4

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8468. Min Act: -0.0000

Data Index: 685965 (The Pile)

Max Activating Token Index: 498

Text #5

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8144. Min Act: -0.0000

Data Index: 1309753 (The Pile)

Max Activating Token Index: 56

Text #6

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8354. Min Act: -0.0000

Data Index: 531993 (The Pile)

Max Activating Token Index: 105

Text #7

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7770. Min Act: -0.0000

Data Index: 1601387 (The Pile)

Max Activating Token Index: 520

Text #8

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.8505. Min Act: -0.0000

Data Index: 1189940 (The Pile)

Max Activating Token Index: 298

Text #9

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7346. Min Act: -0.0000

Data Index: 1846597 (The Pile)

Max Activating Token Index: 567

Text #10

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7689. Min Act: -0.0000

Data Index: 309834 (The Pile)

Max Activating Token Index: 855

Text #11

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7224. Min Act: -0.0000

Data Index: 294044 (The Pile)

Max Activating Token Index: 104

Text #12

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7243. Min Act: -0.0000

Data Index: 792648 (The Pile)

Max Activating Token Index: 588

Text #13

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7058. Min Act: -0.0000

Data Index: 972650 (The Pile)

Max Activating Token Index: 687

Text #14

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.6904. Min Act: -0.0000

Data Index: 408226 (The Pile)

Max Activating Token Index: 371

Text #15

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7101. Min Act: -0.0000

Data Index: 618019 (The Pile)

Max Activating Token Index: 544

Text #16

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.6883. Min Act: -0.0000

Data Index: 688590 (The Pile)

Max Activating Token Index: 700

Text #17

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7274. Min Act: -0.0000

Data Index: 1062799 (The Pile)

Max Activating Token Index: 390

Text #18

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.6858. Min Act: -0.0000

Data Index: 1549239 (The Pile)

Max Activating Token Index: 639

Text #19

Max Range: 0.9863. Min Range: -0.9863

Max Act: 0.7290. Min Act: -0.0000

Data Index: 1978157 (The Pile)

Max Activating Token Index: 474