Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4618 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.8664. Min Act: -0.0000

Data Index: 457051 (The Pile)

Max Activating Token Index: 741

Text #1

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.7699. Min Act: -0.0000

Data Index: 16049 (The Pile)

Max Activating Token Index: 557

Text #2

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.9063. Min Act: -0.0000

Data Index: 584867 (The Pile)

Max Activating Token Index: 78

Text #3

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.4701. Min Act: -0.0000

Data Index: 1008345 (The Pile)

Max Activating Token Index: 271

Text #4

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.4612. Min Act: -0.0000

Data Index: 346802 (The Pile)

Max Activating Token Index: 967

Text #5

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.4909. Min Act: -0.0000

Data Index: 521971 (The Pile)

Max Activating Token Index: 558

Text #6

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.4012. Min Act: -0.0000

Data Index: 436215 (The Pile)

Max Activating Token Index: 151

Text #7

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.5287. Min Act: -0.0000

Data Index: 1622711 (The Pile)

Max Activating Token Index: 565

Text #8

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.2913. Min Act: -0.0000

Data Index: 1527388 (The Pile)

Max Activating Token Index: 255

Text #9

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.1402. Min Act: -0.0000

Data Index: 550870 (The Pile)

Max Activating Token Index: 824

Text #10

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.3542. Min Act: -0.0000

Data Index: 837953 (The Pile)

Max Activating Token Index: 1018

Text #11

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.0298. Min Act: -0.0000

Data Index: 524850 (The Pile)

Max Activating Token Index: 934

Text #12

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.3222. Min Act: -0.0000

Data Index: 1660345 (The Pile)

Max Activating Token Index: 804

Text #13

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.1482. Min Act: -0.0000

Data Index: 651186 (The Pile)

Max Activating Token Index: 174

Text #14

Max Range: 3.9063. Min Range: -3.9063

Max Act: 2.8668. Min Act: -0.0000

Data Index: 981526 (The Pile)

Max Activating Token Index: 58

Text #15

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.2012. Min Act: -0.0000

Data Index: 1656286 (The Pile)

Max Activating Token Index: 416

Text #16

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.3863. Min Act: -0.0000

Data Index: 182450 (The Pile)

Max Activating Token Index: 691

Text #17

Max Range: 3.9063. Min Range: -3.9063

Max Act: 2.7704. Min Act: -0.0000

Data Index: 1482359 (The Pile)

Max Activating Token Index: 166

Text #18

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.0573. Min Act: -0.0000

Data Index: 1511398 (The Pile)

Max Activating Token Index: 287

Text #19

Max Range: 3.9063. Min Range: -3.9063

Max Act: 3.3222. Min Act: -0.0000

Data Index: 701989 (The Pile)

Max Activating Token Index: 639