Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4962 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.9360. Min Act: -0.0000

Data Index: 654258 (The Pile)

Max Activating Token Index: 580

Text #1

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.8997. Min Act: -0.0000

Data Index: 484166 (The Pile)

Max Activating Token Index: 709

Text #2

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.9795. Min Act: -0.0000

Data Index: 1209977 (The Pile)

Max Activating Token Index: 535

Text #3

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.8734. Min Act: -0.0000

Data Index: 788467 (The Pile)

Max Activating Token Index: 644

Text #4

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.9236. Min Act: -0.0000

Data Index: 789130 (The Pile)

Max Activating Token Index: 956

Text #5

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7677. Min Act: -0.0000

Data Index: 1960658 (The Pile)

Max Activating Token Index: 678

Text #6

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.6999. Min Act: -0.0000

Data Index: 1516251 (The Pile)

Max Activating Token Index: 653

Text #7

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.8408. Min Act: -0.0000

Data Index: 1115612 (The Pile)

Max Activating Token Index: 387

Text #8

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.8959. Min Act: -0.0000

Data Index: 1168719 (The Pile)

Max Activating Token Index: 915

Text #9

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7488. Min Act: -0.0000

Data Index: 444696 (The Pile)

Max Activating Token Index: 122

Text #10

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7500. Min Act: -0.0000

Data Index: 1980395 (The Pile)

Max Activating Token Index: 632

Text #11

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7676. Min Act: -0.0000

Data Index: 816460 (The Pile)

Max Activating Token Index: 524

Text #12

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.6985. Min Act: -0.0000

Data Index: 1721435 (The Pile)

Max Activating Token Index: 152

Text #13

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7534. Min Act: -0.0000

Data Index: 180883 (The Pile)

Max Activating Token Index: 652

Text #14

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.6959. Min Act: -0.0000

Data Index: 1319297 (The Pile)

Max Activating Token Index: 243

Text #15

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7074. Min Act: -0.0000

Data Index: 882790 (The Pile)

Max Activating Token Index: 345

Text #16

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7047. Min Act: -0.0000

Data Index: 1112957 (The Pile)

Max Activating Token Index: 575

Text #17

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7061. Min Act: -0.0000

Data Index: 541122 (The Pile)

Max Activating Token Index: 649

Text #18

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7644. Min Act: -0.0000

Data Index: 558027 (The Pile)

Max Activating Token Index: 685

Text #19

Max Range: 0.9795. Min Range: -0.9795

Max Act: 0.7266. Min Act: -0.0000

Data Index: 1383698 (The Pile)

Max Activating Token Index: 855