Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3745 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2568. Min Range: -1.2568

Max Act: 1.2568. Min Act: -0.0000

Data Index: 1724267 (The Pile)

Max Activating Token Index: 512

Text #1

Max Range: 1.2568. Min Range: -1.2568

Max Act: 1.0375. Min Act: -0.0000

Data Index: 259504 (The Pile)

Max Activating Token Index: 216

Text #2

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.9896. Min Act: -0.0000

Data Index: 608303 (The Pile)

Max Activating Token Index: 530

Text #3

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.8369. Min Act: -0.0000

Data Index: 464933 (The Pile)

Max Activating Token Index: 695

Text #4

Max Range: 1.2568. Min Range: -1.2568

Max Act: 1.0371. Min Act: -0.0000

Data Index: 35299 (The Pile)

Max Activating Token Index: 912

Text #5

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.8383. Min Act: -0.0000

Data Index: 633754 (The Pile)

Max Activating Token Index: 40

Text #6

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.7740. Min Act: -0.0000

Data Index: 836420 (The Pile)

Max Activating Token Index: 49

Text #7

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.7855. Min Act: -0.0000

Data Index: 518405 (The Pile)

Max Activating Token Index: 112

Text #8

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.8427. Min Act: -0.0000

Data Index: 1230848 (The Pile)

Max Activating Token Index: 199

Text #9

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.7255. Min Act: -0.0000

Data Index: 1726461 (The Pile)

Max Activating Token Index: 194

Text #10

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6830. Min Act: -0.0000

Data Index: 1273288 (The Pile)

Max Activating Token Index: 309

Text #11

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6677. Min Act: -0.0000

Data Index: 67732 (The Pile)

Max Activating Token Index: 382

Text #12

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.7031. Min Act: -0.0000

Data Index: 1425665 (The Pile)

Max Activating Token Index: 269

Text #13

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.7274. Min Act: -0.0000

Data Index: 1338853 (The Pile)

Max Activating Token Index: 358

Text #14

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6750. Min Act: -0.0000

Data Index: 1076162 (The Pile)

Max Activating Token Index: 205

Text #15

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6127. Min Act: -0.0000

Data Index: 1686512 (The Pile)

Max Activating Token Index: 684

Text #16

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6681. Min Act: -0.0000

Data Index: 1277354 (The Pile)

Max Activating Token Index: 70

Text #17

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6244. Min Act: -0.0000

Data Index: 67328 (The Pile)

Max Activating Token Index: 356

Text #18

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6805. Min Act: -0.0000

Data Index: 1016121 (The Pile)

Max Activating Token Index: 518

Text #19

Max Range: 1.2568. Min Range: -1.2568

Max Act: 0.6560. Min Act: -0.0000

Data Index: 1077331 (The Pile)

Max Activating Token Index: 462