Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2762 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.3923. Min Range: -1.3923

Max Act: 1.3923. Min Act: -0.0000

Data Index: 1663443 (The Pile)

Max Activating Token Index: 556

Text #1

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.9985. Min Act: -0.0000

Data Index: 749812 (The Pile)

Max Activating Token Index: 413

Text #2

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.9480. Min Act: -0.0000

Data Index: 1943461 (The Pile)

Max Activating Token Index: 542

Text #3

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.9685. Min Act: -0.0000

Data Index: 182565 (The Pile)

Max Activating Token Index: 224

Text #4

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8788. Min Act: -0.0000

Data Index: 1114187 (The Pile)

Max Activating Token Index: 338

Text #5

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8277. Min Act: -0.0000

Data Index: 1002768 (The Pile)

Max Activating Token Index: 718

Text #6

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7871. Min Act: -0.0000

Data Index: 706140 (The Pile)

Max Activating Token Index: 964

Text #7

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8430. Min Act: -0.0000

Data Index: 73060 (The Pile)

Max Activating Token Index: 139

Text #8

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8083. Min Act: -0.0000

Data Index: 1122626 (The Pile)

Max Activating Token Index: 869

Text #9

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.6678. Min Act: -0.0000

Data Index: 1673670 (The Pile)

Max Activating Token Index: 86

Text #10

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8310. Min Act: -0.0000

Data Index: 719928 (The Pile)

Max Activating Token Index: 420

Text #11

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8391. Min Act: -0.0000

Data Index: 1896247 (The Pile)

Max Activating Token Index: 638

Text #12

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7837. Min Act: -0.0000

Data Index: 1658445 (The Pile)

Max Activating Token Index: 688

Text #13

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7796. Min Act: -0.0000

Data Index: 1487673 (The Pile)

Max Activating Token Index: 385

Text #14

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7699. Min Act: -0.0000

Data Index: 1510422 (The Pile)

Max Activating Token Index: 194

Text #15

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7630. Min Act: -0.0000

Data Index: 1780004 (The Pile)

Max Activating Token Index: 607

Text #16

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7677. Min Act: -0.0000

Data Index: 894143 (The Pile)

Max Activating Token Index: 30

Text #17

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7443. Min Act: -0.0000

Data Index: 446946 (The Pile)

Max Activating Token Index: 19

Text #18

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.7624. Min Act: -0.0000

Data Index: 1936790 (The Pile)

Max Activating Token Index: 757

Text #19

Max Range: 1.3923. Min Range: -1.3923

Max Act: 0.8549. Min Act: -0.0000

Data Index: 1379412 (The Pile)

Max Activating Token Index: 614