Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1845 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0646. Min Range: -1.0646

Max Act: 1.0646. Min Act: -0.0000

Data Index: 598932 (The Pile)

Max Activating Token Index: 652

Text #1

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8854. Min Act: -0.0000

Data Index: 1710831 (The Pile)

Max Activating Token Index: 705

Text #2

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.9164. Min Act: -0.0000

Data Index: 630872 (The Pile)

Max Activating Token Index: 719

Text #3

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8342. Min Act: -0.0000

Data Index: 1329728 (The Pile)

Max Activating Token Index: 383

Text #4

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.9008. Min Act: -0.0000

Data Index: 1370022 (The Pile)

Max Activating Token Index: 612

Text #5

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8032. Min Act: -0.0000

Data Index: 533085 (The Pile)

Max Activating Token Index: 774

Text #6

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8685. Min Act: -0.0000

Data Index: 408226 (The Pile)

Max Activating Token Index: 370

Text #7

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.9065. Min Act: -0.0000

Data Index: 800897 (The Pile)

Max Activating Token Index: 474

Text #8

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8378. Min Act: -0.0000

Data Index: 597312 (The Pile)

Max Activating Token Index: 616

Text #9

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8248. Min Act: -0.0000

Data Index: 145205 (The Pile)

Max Activating Token Index: 684

Text #10

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8439. Min Act: -0.0000

Data Index: 1663366 (The Pile)

Max Activating Token Index: 226

Text #11

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8350. Min Act: -0.0000

Data Index: 621061 (The Pile)

Max Activating Token Index: 608

Text #12

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7886. Min Act: -0.0000

Data Index: 688590 (The Pile)

Max Activating Token Index: 699

Text #13

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7594. Min Act: -0.0000

Data Index: 1747634 (The Pile)

Max Activating Token Index: 552

Text #14

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7089. Min Act: -0.0000

Data Index: 1928955 (The Pile)

Max Activating Token Index: 146

Text #15

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.8560. Min Act: -0.0000

Data Index: 83968 (The Pile)

Max Activating Token Index: 808

Text #16

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7761. Min Act: -0.0000

Data Index: 1920375 (The Pile)

Max Activating Token Index: 491

Text #17

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7805. Min Act: 0.0000

Data Index: 760966 (The Pile)

Max Activating Token Index: 762

Text #18

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7674. Min Act: -0.0000

Data Index: 867105 (The Pile)

Max Activating Token Index: 970

Text #19

Max Range: 1.0646. Min Range: -1.0646

Max Act: 0.7539. Min Act: -0.0000

Data Index: 108394 (The Pile)

Max Activating Token Index: 282