Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3693 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1147. Min Range: -1.1147

Max Act: 1.1147. Min Act: -0.0000

Data Index: 454711 (The Pile)

Max Activating Token Index: 829

Text #1

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.8211. Min Act: -0.0000

Data Index: 840520 (The Pile)

Max Activating Token Index: 190

Text #2

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.9121. Min Act: -0.0000

Data Index: 703901 (The Pile)

Max Activating Token Index: 735

Text #3

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.8111. Min Act: 0.0001

Data Index: 1526100 (The Pile)

Max Activating Token Index: 208

Text #4

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.7562. Min Act: 0.0001

Data Index: 563506 (The Pile)

Max Activating Token Index: 461

Text #5

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6809. Min Act: -0.0000

Data Index: 313882 (The Pile)

Max Activating Token Index: 537

Text #6

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6706. Min Act: -0.0000

Data Index: 80789 (The Pile)

Max Activating Token Index: 115

Text #7

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6685. Min Act: -0.0000

Data Index: 925325 (The Pile)

Max Activating Token Index: 534

Text #8

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6511. Min Act: -0.0000

Data Index: 1278978 (The Pile)

Max Activating Token Index: 620

Text #9

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6539. Min Act: -0.0000

Data Index: 1398164 (The Pile)

Max Activating Token Index: 867

Text #10

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6391. Min Act: -0.0000

Data Index: 418722 (The Pile)

Max Activating Token Index: 302

Text #11

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6133. Min Act: -0.0000

Data Index: 1283603 (The Pile)

Max Activating Token Index: 38

Text #12

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.6309. Min Act: -0.0000

Data Index: 547498 (The Pile)

Max Activating Token Index: 118

Text #13

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5984. Min Act: 0.0000

Data Index: 1352781 (The Pile)

Max Activating Token Index: 386

Text #14

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5865. Min Act: -0.0000

Data Index: 1387235 (The Pile)

Max Activating Token Index: 238

Text #15

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5493. Min Act: -0.0000

Data Index: 958115 (The Pile)

Max Activating Token Index: 519

Text #16

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5430. Min Act: -0.0000

Data Index: 67981 (The Pile)

Max Activating Token Index: 879

Text #17

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5581. Min Act: 0.0000

Data Index: 1099882 (The Pile)

Max Activating Token Index: 716

Text #18

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5737. Min Act: -0.0000

Data Index: 778363 (The Pile)

Max Activating Token Index: 565

Text #19

Max Range: 1.1147. Min Range: -1.1147

Max Act: 0.5542. Min Act: -0.0000

Data Index: 678072 (The Pile)

Max Activating Token Index: 169