Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3983 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2682. Min Range: -1.2682

Max Act: 1.2206. Min Act: -0.0000

Data Index: 453918 (The Pile)

Max Activating Token Index: 259

Text #1

Max Range: 1.2682. Min Range: -1.2682

Max Act: 1.2682. Min Act: -0.0000

Data Index: 439761 (The Pile)

Max Activating Token Index: 273

Text #2

Max Range: 1.2682. Min Range: -1.2682

Max Act: 1.1156. Min Act: -0.0000

Data Index: 1363023 (The Pile)

Max Activating Token Index: 981

Text #3

Max Range: 1.2682. Min Range: -1.2682

Max Act: 1.1430. Min Act: -0.0000

Data Index: 1845175 (The Pile)

Max Activating Token Index: 786

Text #4

Max Range: 1.2682. Min Range: -1.2682

Max Act: 1.0866. Min Act: -0.0000

Data Index: 361438 (The Pile)

Max Activating Token Index: 878

Text #5

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8567. Min Act: -0.0000

Data Index: 688389 (The Pile)

Max Activating Token Index: 970

Text #6

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.9707. Min Act: -0.0000

Data Index: 20286 (The Pile)

Max Activating Token Index: 89

Text #7

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.9024. Min Act: -0.0000

Data Index: 1679556 (The Pile)

Max Activating Token Index: 20

Text #8

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8772. Min Act: -0.0000

Data Index: 977527 (The Pile)

Max Activating Token Index: 555

Text #9

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8471. Min Act: -0.0000

Data Index: 1354921 (The Pile)

Max Activating Token Index: 42

Text #10

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8744. Min Act: -0.0000

Data Index: 401809 (The Pile)

Max Activating Token Index: 239

Text #11

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8651. Min Act: -0.0000

Data Index: 847728 (The Pile)

Max Activating Token Index: 239

Text #12

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8339. Min Act: -0.0000

Data Index: 1155782 (The Pile)

Max Activating Token Index: 455

Text #13

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8304. Min Act: -0.0000

Data Index: 1508025 (The Pile)

Max Activating Token Index: 107

Text #14

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8254. Min Act: -0.0000

Data Index: 1908102 (The Pile)

Max Activating Token Index: 269

Text #15

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8788. Min Act: -0.0000

Data Index: 904055 (The Pile)

Max Activating Token Index: 331

Text #16

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.7717. Min Act: -0.0000

Data Index: 523671 (The Pile)

Max Activating Token Index: 420

Text #17

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8499. Min Act: -0.0000

Data Index: 182103 (The Pile)

Max Activating Token Index: 604

Text #18

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.8537. Min Act: -0.0000

Data Index: 1844007 (The Pile)

Max Activating Token Index: 81

Text #19

Max Range: 1.2682. Min Range: -1.2682

Max Act: 0.9879. Min Act: -0.0000

Data Index: 1716419 (The Pile)

Max Activating Token Index: 978