Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 869 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.2143. Min Act: -0.0000

Data Index: 1419419 (The Pile)

Max Activating Token Index: 27

Text #1

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.2403. Min Act: -0.0000

Data Index: 1981282 (The Pile)

Max Activating Token Index: 77

Text #2

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.2028. Min Act: -0.0000

Data Index: 876121 (The Pile)

Max Activating Token Index: 172

Text #3

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.1303. Min Act: -0.0000

Data Index: 1457361 (The Pile)

Max Activating Token Index: 779

Text #4

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.1185. Min Act: -0.0000

Data Index: 846048 (The Pile)

Max Activating Token Index: 803

Text #5

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.0950. Min Act: -0.0000

Data Index: 34956 (The Pile)

Max Activating Token Index: 364

Text #6

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.1447. Min Act: -0.0000

Data Index: 1572484 (The Pile)

Max Activating Token Index: 455

Text #7

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.1461. Min Act: -0.0000

Data Index: 1193146 (The Pile)

Max Activating Token Index: 74

Text #8

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.0680. Min Act: -0.0000

Data Index: 1931905 (The Pile)

Max Activating Token Index: 818

Text #9

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.0719. Min Act: -0.0000

Data Index: 1102453 (The Pile)

Max Activating Token Index: 497

Text #10

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9953. Min Act: -0.0000

Data Index: 936839 (The Pile)

Max Activating Token Index: 807

Text #11

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9191. Min Act: -0.0000

Data Index: 4520 (The Pile)

Max Activating Token Index: 383

Text #12

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9059. Min Act: -0.0000

Data Index: 144548 (The Pile)

Max Activating Token Index: 100

Text #13

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9690. Min Act: -0.0000

Data Index: 1260622 (The Pile)

Max Activating Token Index: 910

Text #14

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9628. Min Act: -0.0000

Data Index: 1897153 (The Pile)

Max Activating Token Index: 206

Text #15

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9600. Min Act: -0.0000

Data Index: 1155704 (The Pile)

Max Activating Token Index: 528

Text #16

Max Range: 1.2403. Min Range: -1.2403

Max Act: 1.1020. Min Act: -0.0000

Data Index: 300354 (The Pile)

Max Activating Token Index: 888

Text #17

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.9318. Min Act: -0.0000

Data Index: 1117286 (The Pile)

Max Activating Token Index: 884

Text #18

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.8576. Min Act: -0.0000

Data Index: 1608058 (The Pile)

Max Activating Token Index: 951

Text #19

Max Range: 1.2403. Min Range: -1.2403

Max Act: 0.8396. Min Act: -0.0000

Data Index: 739698 (The Pile)

Max Activating Token Index: 552