Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 279 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.1508. Min Range: -1.1508

Max Act: 1.1508. Min Act: -0.0000

Data Index: 1369676 (The Pile)

Max Activating Token Index: 985

Text #1

Max Range: 1.1508. Min Range: -1.1508

Max Act: 1.1317. Min Act: -0.0000

Data Index: 699917 (The Pile)

Max Activating Token Index: 175

Text #2

Max Range: 1.1508. Min Range: -1.1508

Max Act: 1.0479. Min Act: -0.0000

Data Index: 1426295 (The Pile)

Max Activating Token Index: 542

Text #3

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9764. Min Act: -0.0000

Data Index: 1157212 (The Pile)

Max Activating Token Index: 222

Text #4

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9830. Min Act: -0.0000

Data Index: 287087 (The Pile)

Max Activating Token Index: 212

Text #5

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9510. Min Act: -0.0000

Data Index: 1239299 (The Pile)

Max Activating Token Index: 941

Text #6

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9211. Min Act: -0.0000

Data Index: 205003 (The Pile)

Max Activating Token Index: 194

Text #7

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9785. Min Act: -0.0000

Data Index: 1771513 (The Pile)

Max Activating Token Index: 113

Text #8

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8880. Min Act: -0.0000

Data Index: 516990 (The Pile)

Max Activating Token Index: 359

Text #9

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9623. Min Act: -0.0000

Data Index: 1586601 (The Pile)

Max Activating Token Index: 244

Text #10

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9817. Min Act: -0.0000

Data Index: 541408 (The Pile)

Max Activating Token Index: 964

Text #11

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9252. Min Act: -0.0000

Data Index: 906055 (The Pile)

Max Activating Token Index: 1007

Text #12

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8373. Min Act: -0.0000

Data Index: 123183 (The Pile)

Max Activating Token Index: 483

Text #13

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8462. Min Act: -0.0000

Data Index: 19746 (The Pile)

Max Activating Token Index: 940

Text #14

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9388. Min Act: -0.0000

Data Index: 707518 (The Pile)

Max Activating Token Index: 707

Text #15

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8657. Min Act: -0.0000

Data Index: 359945 (The Pile)

Max Activating Token Index: 190

Text #16

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8837. Min Act: -0.0000

Data Index: 642343 (The Pile)

Max Activating Token Index: 110

Text #17

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9935. Min Act: -0.0000

Data Index: 1243257 (The Pile)

Max Activating Token Index: 95

Text #18

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.9095. Min Act: -0.0000

Data Index: 950085 (The Pile)

Max Activating Token Index: 101

Text #19

Max Range: 1.1508. Min Range: -1.1508

Max Act: 0.8559. Min Act: -0.0000

Data Index: 1739937 (The Pile)

Max Activating Token Index: 538