Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2685 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.6430. Min Act: -0.0000

Data Index: 1775193 (The Pile)

Max Activating Token Index: 989

Text #1

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.4950. Min Act: -0.0000

Data Index: 309754 (The Pile)

Max Activating Token Index: 968

Text #2

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.4151. Min Act: -0.0000

Data Index: 1704045 (The Pile)

Max Activating Token Index: 350

Text #3

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.4064. Min Act: -0.0000

Data Index: 980684 (The Pile)

Max Activating Token Index: 713

Text #4

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3998. Min Act: -0.0000

Data Index: 208102 (The Pile)

Max Activating Token Index: 397

Text #5

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3872. Min Act: -0.0000

Data Index: 1717532 (The Pile)

Max Activating Token Index: 207

Text #6

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3748. Min Act: -0.0000

Data Index: 1227871 (The Pile)

Max Activating Token Index: 366

Text #7

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3992. Min Act: -0.0000

Data Index: 1562965 (The Pile)

Max Activating Token Index: 280

Text #8

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3550. Min Act: -0.0000

Data Index: 433754 (The Pile)

Max Activating Token Index: 898

Text #9

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3710. Min Act: -0.0000

Data Index: 1615043 (The Pile)

Max Activating Token Index: 177

Text #10

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3511. Min Act: -0.0000

Data Index: 1286437 (The Pile)

Max Activating Token Index: 665

Text #11

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3533. Min Act: -0.0000

Data Index: 429103 (The Pile)

Max Activating Token Index: 822

Text #12

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3195. Min Act: -0.0000

Data Index: 1556065 (The Pile)

Max Activating Token Index: 711

Text #13

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3244. Min Act: -0.0000

Data Index: 1373711 (The Pile)

Max Activating Token Index: 219

Text #14

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.2944. Min Act: -0.0000

Data Index: 1707337 (The Pile)

Max Activating Token Index: 1015

Text #15

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.2903. Min Act: -0.0000

Data Index: 1747254 (The Pile)

Max Activating Token Index: 977

Text #16

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.2808. Min Act: -0.0000

Data Index: 1357140 (The Pile)

Max Activating Token Index: 121

Text #17

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3039. Min Act: -0.0000

Data Index: 478838 (The Pile)

Max Activating Token Index: 284

Text #18

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.2977. Min Act: -0.0000

Data Index: 1108172 (The Pile)

Max Activating Token Index: 943

Text #19

Max Range: 0.6430. Min Range: -0.6430

Max Act: 0.3194. Min Act: -0.0000

Data Index: 1133197 (The Pile)

Max Activating Token Index: 277