Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2051 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.4565. Min Act: -0.0000

Data Index: 324301 (The Pile)

Max Activating Token Index: 947

Text #1

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.3817. Min Act: -0.0000

Data Index: 1756080 (The Pile)

Max Activating Token Index: 808

Text #2

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.0290. Min Act: -0.0000

Data Index: 724334 (The Pile)

Max Activating Token Index: 27

Text #3

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.0745. Min Act: -0.0000

Data Index: 87085 (The Pile)

Max Activating Token Index: 474

Text #4

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9523. Min Act: -0.0000

Data Index: 1932759 (The Pile)

Max Activating Token Index: 137

Text #5

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.0120. Min Act: -0.0000

Data Index: 1210583 (The Pile)

Max Activating Token Index: 594

Text #6

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9953. Min Act: 0.0000

Data Index: 1152923 (The Pile)

Max Activating Token Index: 468

Text #7

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.0442. Min Act: -0.0000

Data Index: 122806 (The Pile)

Max Activating Token Index: 57

Text #8

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9676. Min Act: -0.0000

Data Index: 1775848 (The Pile)

Max Activating Token Index: 220

Text #9

Max Range: 1.4565. Min Range: -1.4565

Max Act: 1.0277. Min Act: -0.0000

Data Index: 599794 (The Pile)

Max Activating Token Index: 436

Text #10

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8996. Min Act: -0.0000

Data Index: 1567548 (The Pile)

Max Activating Token Index: 911

Text #11

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9625. Min Act: -0.0000

Data Index: 1531221 (The Pile)

Max Activating Token Index: 696

Text #12

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9287. Min Act: -0.0000

Data Index: 674460 (The Pile)

Max Activating Token Index: 258

Text #13

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9772. Min Act: -0.0000

Data Index: 1259812 (The Pile)

Max Activating Token Index: 88

Text #14

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8316. Min Act: -0.0000

Data Index: 38716 (The Pile)

Max Activating Token Index: 910

Text #15

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8320. Min Act: -0.0000

Data Index: 1227012 (The Pile)

Max Activating Token Index: 1014

Text #16

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.9333. Min Act: -0.0000

Data Index: 1086364 (The Pile)

Max Activating Token Index: 356

Text #17

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8335. Min Act: -0.0000

Data Index: 941349 (The Pile)

Max Activating Token Index: 462

Text #18

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8771. Min Act: -0.0000

Data Index: 1634581 (The Pile)

Max Activating Token Index: 60

Text #19

Max Range: 1.4565. Min Range: -1.4565

Max Act: 0.8428. Min Act: -0.0000

Data Index: 1801768 (The Pile)

Max Activating Token Index: 1010