Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2948 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5346. Min Range: -3.5346

Max Act: 3.5346. Min Act: -0.0000

Data Index: 1402960 (The Pile)

Max Activating Token Index: 748

Text #1

Max Range: 3.5346. Min Range: -3.5346

Max Act: 3.2292. Min Act: -0.0000

Data Index: 315070 (The Pile)

Max Activating Token Index: 463

Text #2

Max Range: 3.5346. Min Range: -3.5346

Max Act: 3.1230. Min Act: -0.0000

Data Index: 709984 (The Pile)

Max Activating Token Index: 337

Text #3

Max Range: 3.5346. Min Range: -3.5346

Max Act: 3.1625. Min Act: -0.0000

Data Index: 402645 (The Pile)

Max Activating Token Index: 126

Text #4

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.7590. Min Act: -0.0000

Data Index: 1006394 (The Pile)

Max Activating Token Index: 708

Text #5

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.2994. Min Act: -0.0000

Data Index: 146001 (The Pile)

Max Activating Token Index: 998

Text #6

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.6420. Min Act: -0.0000

Data Index: 1465670 (The Pile)

Max Activating Token Index: 583

Text #7

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.3601. Min Act: -0.0000

Data Index: 761338 (The Pile)

Max Activating Token Index: 291

Text #8

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.2866. Min Act: -0.0000

Data Index: 512866 (The Pile)

Max Activating Token Index: 538

Text #9

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.2811. Min Act: -0.0000

Data Index: 1241066 (The Pile)

Max Activating Token Index: 528

Text #10

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.2311. Min Act: -0.0000

Data Index: 1556659 (The Pile)

Max Activating Token Index: 359

Text #11

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.2078. Min Act: -0.0000

Data Index: 1169682 (The Pile)

Max Activating Token Index: 559

Text #12

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.1549. Min Act: -0.0000

Data Index: 1178165 (The Pile)

Max Activating Token Index: 843

Text #13

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.1335. Min Act: -0.0000

Data Index: 278518 (The Pile)

Max Activating Token Index: 110

Text #14

Max Range: 3.5346. Min Range: -3.5346

Max Act: 1.8897. Min Act: -0.0000

Data Index: 244680 (The Pile)

Max Activating Token Index: 275

Text #15

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.1989. Min Act: -0.0000

Data Index: 1500731 (The Pile)

Max Activating Token Index: 944

Text #16

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.0322. Min Act: -0.0000

Data Index: 1551869 (The Pile)

Max Activating Token Index: 313

Text #17

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.0497. Min Act: -0.0000

Data Index: 1305140 (The Pile)

Max Activating Token Index: 381

Text #18

Max Range: 3.5346. Min Range: -3.5346

Max Act: 2.4063. Min Act: -0.0000

Data Index: 1716289 (The Pile)

Max Activating Token Index: 980

Text #19

Max Range: 3.5346. Min Range: -3.5346

Max Act: 1.7876. Min Act: -0.0000

Data Index: 1566899 (The Pile)

Max Activating Token Index: 57