Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 274 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.4638. Min Act: 0.0000

Data Index: 115446 (The Pile)

Max Activating Token Index: 228

Text #1

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.4295. Min Act: -0.0000

Data Index: 794908 (The Pile)

Max Activating Token Index: 455

Text #2

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.4362. Min Act: -0.0000

Data Index: 526163 (The Pile)

Max Activating Token Index: 450

Text #3

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.4284. Min Act: -0.0000

Data Index: 1954927 (The Pile)

Max Activating Token Index: 947

Text #4

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.4324. Min Act: -0.0000

Data Index: 455040 (The Pile)

Max Activating Token Index: 739

Text #5

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3760. Min Act: -0.0000

Data Index: 87507 (The Pile)

Max Activating Token Index: 778

Text #6

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3421. Min Act: -0.0000

Data Index: 1767210 (The Pile)

Max Activating Token Index: 847

Text #7

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3575. Min Act: -0.0000

Data Index: 1966342 (The Pile)

Max Activating Token Index: 531

Text #8

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3867. Min Act: -0.0000

Data Index: 1942022 (The Pile)

Max Activating Token Index: 22

Text #9

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3371. Min Act: -0.0000

Data Index: 978932 (The Pile)

Max Activating Token Index: 339

Text #10

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3394. Min Act: -0.0000

Data Index: 1341201 (The Pile)

Max Activating Token Index: 703

Text #11

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3596. Min Act: -0.0000

Data Index: 1187322 (The Pile)

Max Activating Token Index: 43

Text #12

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3378. Min Act: -0.0000

Data Index: 158358 (The Pile)

Max Activating Token Index: 183

Text #13

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3518. Min Act: -0.0000

Data Index: 1586180 (The Pile)

Max Activating Token Index: 217

Text #14

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3566. Min Act: -0.0000

Data Index: 532293 (The Pile)

Max Activating Token Index: 287

Text #15

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3033. Min Act: -0.0000

Data Index: 1727340 (The Pile)

Max Activating Token Index: 638

Text #16

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3444. Min Act: -0.0000

Data Index: 965258 (The Pile)

Max Activating Token Index: 375

Text #17

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3888. Min Act: -0.0000

Data Index: 522129 (The Pile)

Max Activating Token Index: 553

Text #18

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3404. Min Act: -0.0000

Data Index: 1466385 (The Pile)

Max Activating Token Index: 361

Text #19

Max Range: 0.4638. Min Range: -0.4638

Max Act: 0.3494. Min Act: -0.0000

Data Index: 1427576 (The Pile)

Max Activating Token Index: 958