Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 710 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.6199. Min Act: -0.0000

Data Index: 1180642 (The Pile)

Max Activating Token Index: 990

Text #1

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.5034. Min Act: -0.0000

Data Index: 364091 (The Pile)

Max Activating Token Index: 786

Text #2

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.5117. Min Act: -0.0000

Data Index: 357196 (The Pile)

Max Activating Token Index: 603

Text #3

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4966. Min Act: -0.0000

Data Index: 1035018 (The Pile)

Max Activating Token Index: 616

Text #4

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4542. Min Act: -0.0000

Data Index: 1654656 (The Pile)

Max Activating Token Index: 785

Text #5

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4347. Min Act: -0.0000

Data Index: 591205 (The Pile)

Max Activating Token Index: 958

Text #6

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4191. Min Act: -0.0000

Data Index: 1617623 (The Pile)

Max Activating Token Index: 925

Text #7

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4185. Min Act: -0.0000

Data Index: 457495 (The Pile)

Max Activating Token Index: 846

Text #8

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4459. Min Act: -0.0000

Data Index: 766454 (The Pile)

Max Activating Token Index: 34

Text #9

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3899. Min Act: -0.0000

Data Index: 93356 (The Pile)

Max Activating Token Index: 503

Text #10

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4211. Min Act: -0.0000

Data Index: 32731 (The Pile)

Max Activating Token Index: 408

Text #11

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4394. Min Act: -0.0000

Data Index: 677557 (The Pile)

Max Activating Token Index: 548

Text #12

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3996. Min Act: -0.0000

Data Index: 448067 (The Pile)

Max Activating Token Index: 179

Text #13

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4019. Min Act: -0.0000

Data Index: 603082 (The Pile)

Max Activating Token Index: 908

Text #14

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4075. Min Act: -0.0000

Data Index: 546858 (The Pile)

Max Activating Token Index: 679

Text #15

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3709. Min Act: -0.0000

Data Index: 90400 (The Pile)

Max Activating Token Index: 771

Text #16

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3877. Min Act: -0.0000

Data Index: 1381267 (The Pile)

Max Activating Token Index: 216

Text #17

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3829. Min Act: -0.0000

Data Index: 1132234 (The Pile)

Max Activating Token Index: 370

Text #18

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.3768. Min Act: -0.0000

Data Index: 397904 (The Pile)

Max Activating Token Index: 956

Text #19

Max Range: 0.6199. Min Range: -0.6199

Max Act: 0.4132. Min Act: -0.0000

Data Index: 1304071 (The Pile)

Max Activating Token Index: 557