Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 534 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.9954. Min Act: -0.0000

Data Index: 146178 (The Pile)

Max Activating Token Index: 100

Text #1

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.9622. Min Act: -0.0000

Data Index: 1459034 (The Pile)

Max Activating Token Index: 292

Text #2

Max Range: 1.0118. Min Range: -1.0118

Max Act: 1.0118. Min Act: -0.0000

Data Index: 105586 (The Pile)

Max Activating Token Index: 128

Text #3

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8874. Min Act: -0.0000

Data Index: 1113238 (The Pile)

Max Activating Token Index: 377

Text #4

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.9102. Min Act: -0.0000

Data Index: 603037 (The Pile)

Max Activating Token Index: 226

Text #5

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.9030. Min Act: -0.0000

Data Index: 1047566 (The Pile)

Max Activating Token Index: 32

Text #6

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8470. Min Act: -0.0000

Data Index: 999521 (The Pile)

Max Activating Token Index: 563

Text #7

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8873. Min Act: -0.0000

Data Index: 1059085 (The Pile)

Max Activating Token Index: 58

Text #8

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8860. Min Act: -0.0000

Data Index: 361765 (The Pile)

Max Activating Token Index: 319

Text #9

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8940. Min Act: -0.0000

Data Index: 872545 (The Pile)

Max Activating Token Index: 83

Text #10

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8590. Min Act: -0.0000

Data Index: 165148 (The Pile)

Max Activating Token Index: 671

Text #11

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8064. Min Act: -0.0000

Data Index: 1426375 (The Pile)

Max Activating Token Index: 94

Text #12

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8074. Min Act: -0.0000

Data Index: 646666 (The Pile)

Max Activating Token Index: 414

Text #13

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8455. Min Act: -0.0000

Data Index: 1309726 (The Pile)

Max Activating Token Index: 738

Text #14

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8464. Min Act: -0.0000

Data Index: 286885 (The Pile)

Max Activating Token Index: 360

Text #15

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.9000. Min Act: -0.0000

Data Index: 784931 (The Pile)

Max Activating Token Index: 631

Text #16

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8922. Min Act: -0.0000

Data Index: 1776524 (The Pile)

Max Activating Token Index: 166

Text #17

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8888. Min Act: -0.0000

Data Index: 1413467 (The Pile)

Max Activating Token Index: 197

Text #18

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8775. Min Act: -0.0000

Data Index: 410831 (The Pile)

Max Activating Token Index: 928

Text #19

Max Range: 1.0118. Min Range: -1.0118

Max Act: 0.8327. Min Act: -0.0000

Data Index: 949722 (The Pile)

Max Activating Token Index: 519