Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2076 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.8662. Min Act: -0.0000

Data Index: 680227 (The Pile)

Max Activating Token Index: 202

Text #1

Max Range: 1.0241. Min Range: -1.0241

Max Act: 1.0241. Min Act: -0.0000

Data Index: 1412150 (The Pile)

Max Activating Token Index: 317

Text #2

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.8866. Min Act: -0.0000

Data Index: 1419237 (The Pile)

Max Activating Token Index: 99

Text #3

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.8490. Min Act: -0.0000

Data Index: 1761782 (The Pile)

Max Activating Token Index: 695

Text #4

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.8775. Min Act: -0.0000

Data Index: 803698 (The Pile)

Max Activating Token Index: 891

Text #5

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7868. Min Act: -0.0000

Data Index: 1735511 (The Pile)

Max Activating Token Index: 399

Text #6

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7886. Min Act: -0.0000

Data Index: 213114 (The Pile)

Max Activating Token Index: 634

Text #7

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7659. Min Act: -0.0000

Data Index: 1165072 (The Pile)

Max Activating Token Index: 507

Text #8

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7727. Min Act: -0.0000

Data Index: 1406626 (The Pile)

Max Activating Token Index: 113

Text #9

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7467. Min Act: -0.0000

Data Index: 1593919 (The Pile)

Max Activating Token Index: 883

Text #10

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7451. Min Act: -0.0000

Data Index: 860828 (The Pile)

Max Activating Token Index: 166

Text #11

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.6812. Min Act: -0.0000

Data Index: 1587421 (The Pile)

Max Activating Token Index: 939

Text #12

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7156. Min Act: -0.0000

Data Index: 1777309 (The Pile)

Max Activating Token Index: 335

Text #13

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7140. Min Act: -0.0000

Data Index: 17059 (The Pile)

Max Activating Token Index: 387

Text #14

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7031. Min Act: -0.0000

Data Index: 1646451 (The Pile)

Max Activating Token Index: 463

Text #15

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.7082. Min Act: -0.0000

Data Index: 914393 (The Pile)

Max Activating Token Index: 442

Text #16

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.6621. Min Act: -0.0000

Data Index: 24846 (The Pile)

Max Activating Token Index: 192

Text #17

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.6821. Min Act: -0.0000

Data Index: 1881057 (The Pile)

Max Activating Token Index: 755

Text #18

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.6690. Min Act: -0.0000

Data Index: 1080604 (The Pile)

Max Activating Token Index: 837

Text #19

Max Range: 1.0241. Min Range: -1.0241

Max Act: 0.6928. Min Act: -0.0000

Data Index: 901102 (The Pile)

Max Activating Token Index: 505