Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 707 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.5868. Min Act: -0.0000

Data Index: 847763 (The Pile)

Max Activating Token Index: 343

Text #1

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4501. Min Act: -0.0000

Data Index: 1528047 (The Pile)

Max Activating Token Index: 999

Text #2

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4647. Min Act: -0.0000

Data Index: 1102082 (The Pile)

Max Activating Token Index: 71

Text #3

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.3896. Min Act: -0.0000

Data Index: 1511807 (The Pile)

Max Activating Token Index: 268

Text #4

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4311. Min Act: -0.0000

Data Index: 1033102 (The Pile)

Max Activating Token Index: 5

Text #5

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4465. Min Act: -0.0000

Data Index: 1088489 (The Pile)

Max Activating Token Index: 418

Text #6

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4035. Min Act: -0.0000

Data Index: 683864 (The Pile)

Max Activating Token Index: 295

Text #7

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.4653. Min Act: -0.0000

Data Index: 757393 (The Pile)

Max Activating Token Index: 674

Text #8

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2692. Min Act: -0.0000

Data Index: 333771 (The Pile)

Max Activating Token Index: 893

Text #9

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.3440. Min Act: -0.0000

Data Index: 272345 (The Pile)

Max Activating Token Index: 105

Text #10

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.3236. Min Act: -0.0000

Data Index: 383349 (The Pile)

Max Activating Token Index: 462

Text #11

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2649. Min Act: -0.0000

Data Index: 482352 (The Pile)

Max Activating Token Index: 195

Text #12

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.1247. Min Act: -0.0000

Data Index: 1272447 (The Pile)

Max Activating Token Index: 1016

Text #13

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.3484. Min Act: -0.0000

Data Index: 424960 (The Pile)

Max Activating Token Index: 1017

Text #14

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2274. Min Act: -0.0000

Data Index: 941829 (The Pile)

Max Activating Token Index: 5

Text #15

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.1663. Min Act: -0.0000

Data Index: 888387 (The Pile)

Max Activating Token Index: 964

Text #16

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2929. Min Act: -0.0000

Data Index: 1529937 (The Pile)

Max Activating Token Index: 751

Text #17

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2469. Min Act: -0.0000

Data Index: 322709 (The Pile)

Max Activating Token Index: 67

Text #18

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.2058. Min Act: -0.0000

Data Index: 610750 (The Pile)

Max Activating Token Index: 335

Text #19

Max Range: 1.5868. Min Range: -1.5868

Max Act: 1.1692. Min Act: -0.0000

Data Index: 1168852 (The Pile)

Max Activating Token Index: 114