Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1075 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9872. Min Act: -0.0000

Data Index: 1408266 (The Pile)

Max Activating Token Index: 825

Text #1

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9626. Min Act: -0.0000

Data Index: 1014890 (The Pile)

Max Activating Token Index: 405

Text #2

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9524. Min Act: -0.0000

Data Index: 498837 (The Pile)

Max Activating Token Index: 121

Text #3

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9091. Min Act: -0.0000

Data Index: 1679271 (The Pile)

Max Activating Token Index: 340

Text #4

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8873. Min Act: -0.0000

Data Index: 407762 (The Pile)

Max Activating Token Index: 768

Text #5

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8853. Min Act: -0.0000

Data Index: 1879962 (The Pile)

Max Activating Token Index: 200

Text #6

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9347. Min Act: -0.0000

Data Index: 491200 (The Pile)

Max Activating Token Index: 811

Text #7

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8585. Min Act: -0.0000

Data Index: 1129894 (The Pile)

Max Activating Token Index: 287

Text #8

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8040. Min Act: -0.0000

Data Index: 1847104 (The Pile)

Max Activating Token Index: 605

Text #9

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8456. Min Act: -0.0000

Data Index: 189077 (The Pile)

Max Activating Token Index: 556

Text #10

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9228. Min Act: -0.0000

Data Index: 763890 (The Pile)

Max Activating Token Index: 580

Text #11

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9068. Min Act: -0.0000

Data Index: 1030082 (The Pile)

Max Activating Token Index: 169

Text #12

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.9240. Min Act: -0.0000

Data Index: 1192442 (The Pile)

Max Activating Token Index: 746

Text #13

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8688. Min Act: -0.0000

Data Index: 1890381 (The Pile)

Max Activating Token Index: 534

Text #14

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8688. Min Act: -0.0000

Data Index: 152546 (The Pile)

Max Activating Token Index: 22

Text #15

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.7781. Min Act: -0.0000

Data Index: 1357812 (The Pile)

Max Activating Token Index: 151

Text #16

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8349. Min Act: -0.0000

Data Index: 430411 (The Pile)

Max Activating Token Index: 508

Text #17

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.7919. Min Act: -0.0000

Data Index: 164341 (The Pile)

Max Activating Token Index: 172

Text #18

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8405. Min Act: -0.0000

Data Index: 1981680 (The Pile)

Max Activating Token Index: 84

Text #19

Max Range: 0.9872. Min Range: -0.9872

Max Act: 0.8134. Min Act: -0.0000

Data Index: 468658 (The Pile)

Max Activating Token Index: 867