Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 851 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1701. Min Range: -3.1701

Max Act: 3.1701. Min Act: -0.0000

Data Index: 797177 (The Pile)

Max Activating Token Index: 1012

Text #1

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.7086. Min Act: -0.0000

Data Index: 372396 (The Pile)

Max Activating Token Index: 322

Text #2

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.5096. Min Act: -0.0000

Data Index: 1042567 (The Pile)

Max Activating Token Index: 141

Text #3

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.4554. Min Act: -0.0000

Data Index: 1663453 (The Pile)

Max Activating Token Index: 427

Text #4

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.4541. Min Act: -0.0000

Data Index: 1374237 (The Pile)

Max Activating Token Index: 450

Text #5

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.5795. Min Act: -0.0000

Data Index: 416128 (The Pile)

Max Activating Token Index: 917

Text #6

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.8102. Min Act: -0.0000

Data Index: 719980 (The Pile)

Max Activating Token Index: 923

Text #7

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0536. Min Act: -0.0000

Data Index: 157314 (The Pile)

Max Activating Token Index: 183

Text #8

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.2609. Min Act: -0.0000

Data Index: 141605 (The Pile)

Max Activating Token Index: 470

Text #9

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.1055. Min Act: -0.0000

Data Index: 1255322 (The Pile)

Max Activating Token Index: 150

Text #10

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.3574. Min Act: -0.0000

Data Index: 1485686 (The Pile)

Max Activating Token Index: 651

Text #11

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0594. Min Act: -0.0000

Data Index: 1701753 (The Pile)

Max Activating Token Index: 1000

Text #12

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0442. Min Act: -0.0000

Data Index: 1057218 (The Pile)

Max Activating Token Index: 80

Text #13

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0672. Min Act: -0.0000

Data Index: 952620 (The Pile)

Max Activating Token Index: 372

Text #14

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.2972. Min Act: -0.0000

Data Index: 689539 (The Pile)

Max Activating Token Index: 247

Text #15

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.7355. Min Act: -0.0000

Data Index: 1064350 (The Pile)

Max Activating Token Index: 893

Text #16

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.9426. Min Act: -0.0000

Data Index: 1820337 (The Pile)

Max Activating Token Index: 351

Text #17

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.7956. Min Act: -0.0000

Data Index: 256293 (The Pile)

Max Activating Token Index: 871

Text #18

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0206. Min Act: -0.0000

Data Index: 126670 (The Pile)

Max Activating Token Index: 426

Text #19

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.8905. Min Act: -0.0000

Data Index: 254814 (The Pile)

Max Activating Token Index: 34