Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2306 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1276. Min Range: -1.1276

Max Act: 1.1276. Min Act: -0.0000

Data Index: 1749099 (The Pile)

Max Activating Token Index: 398

Text #1

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7717. Min Act: -0.0000

Data Index: 52938 (The Pile)

Max Activating Token Index: 357

Text #2

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7655. Min Act: -0.0000

Data Index: 873276 (The Pile)

Max Activating Token Index: 416

Text #3

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7113. Min Act: -0.0000

Data Index: 1030232 (The Pile)

Max Activating Token Index: 334

Text #4

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6811. Min Act: -0.0000

Data Index: 760236 (The Pile)

Max Activating Token Index: 819

Text #5

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6655. Min Act: -0.0000

Data Index: 1274078 (The Pile)

Max Activating Token Index: 124

Text #6

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6599. Min Act: -0.0000

Data Index: 1816488 (The Pile)

Max Activating Token Index: 470

Text #7

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6606. Min Act: -0.0000

Data Index: 163052 (The Pile)

Max Activating Token Index: 381

Text #8

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7296. Min Act: -0.0000

Data Index: 1247005 (The Pile)

Max Activating Token Index: 422

Text #9

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6592. Min Act: -0.0000

Data Index: 73803 (The Pile)

Max Activating Token Index: 402

Text #10

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7171. Min Act: -0.0000

Data Index: 630196 (The Pile)

Max Activating Token Index: 220

Text #11

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.7158. Min Act: -0.0000

Data Index: 1251501 (The Pile)

Max Activating Token Index: 370

Text #12

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6700. Min Act: -0.0000

Data Index: 1692968 (The Pile)

Max Activating Token Index: 66

Text #13

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6583. Min Act: -0.0000

Data Index: 725453 (The Pile)

Max Activating Token Index: 485

Text #14

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6770. Min Act: -0.0000

Data Index: 1616070 (The Pile)

Max Activating Token Index: 167

Text #15

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6886. Min Act: -0.0000

Data Index: 363186 (The Pile)

Max Activating Token Index: 882

Text #16

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.5928. Min Act: -0.0000

Data Index: 662012 (The Pile)

Max Activating Token Index: 598

Text #17

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6716. Min Act: -0.0000

Data Index: 534069 (The Pile)

Max Activating Token Index: 889

Text #18

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6707. Min Act: -0.0000

Data Index: 808371 (The Pile)

Max Activating Token Index: 296

Text #19

Max Range: 1.1276. Min Range: -1.1276

Max Act: 0.6706. Min Act: -0.0000

Data Index: 52931 (The Pile)

Max Activating Token Index: 713