Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3163 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.7627. Min Act: -0.0000

Data Index: 619854 (The Pile)

Max Activating Token Index: 142

Text #1

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5704. Min Act: -0.0000

Data Index: 1759820 (The Pile)

Max Activating Token Index: 769

Text #2

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5633. Min Act: -0.0000

Data Index: 469785 (The Pile)

Max Activating Token Index: 809

Text #3

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5590. Min Act: -0.0000

Data Index: 181740 (The Pile)

Max Activating Token Index: 297

Text #4

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5445. Min Act: -0.0000

Data Index: 1339102 (The Pile)

Max Activating Token Index: 559

Text #5

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5445. Min Act: -0.0000

Data Index: 364811 (The Pile)

Max Activating Token Index: 825

Text #6

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.4621. Min Act: -0.0000

Data Index: 1636597 (The Pile)

Max Activating Token Index: 757

Text #7

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5351. Min Act: -0.0000

Data Index: 521355 (The Pile)

Max Activating Token Index: 926

Text #8

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5160. Min Act: -0.0000

Data Index: 1033192 (The Pile)

Max Activating Token Index: 737

Text #9

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5098. Min Act: -0.0000

Data Index: 1211266 (The Pile)

Max Activating Token Index: 444

Text #10

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5984. Min Act: -0.0000

Data Index: 719133 (The Pile)

Max Activating Token Index: 1017

Text #11

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.4393. Min Act: -0.0000

Data Index: 332772 (The Pile)

Max Activating Token Index: 610

Text #12

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5817. Min Act: -0.0000

Data Index: 805003 (The Pile)

Max Activating Token Index: 143

Text #13

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5004. Min Act: -0.0000

Data Index: 1249258 (The Pile)

Max Activating Token Index: 969

Text #14

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5027. Min Act: -0.0000

Data Index: 1326829 (The Pile)

Max Activating Token Index: 947

Text #15

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5635. Min Act: -0.0000

Data Index: 286226 (The Pile)

Max Activating Token Index: 753

Text #16

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.4636. Min Act: -0.0000

Data Index: 1231039 (The Pile)

Max Activating Token Index: 328

Text #17

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.4604. Min Act: -0.0000

Data Index: 121826 (The Pile)

Max Activating Token Index: 259

Text #18

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.5383. Min Act: -0.0000

Data Index: 1066170 (The Pile)

Max Activating Token Index: 970

Text #19

Max Range: 1.7627. Min Range: -1.7627

Max Act: 1.3676. Min Act: -0.0000

Data Index: 1557026 (The Pile)

Max Activating Token Index: 842