Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 187 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.8118. Min Act: -0.0000

Data Index: 1960777 (The Pile)

Max Activating Token Index: 387

Text #1

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.7087. Min Act: -0.0000

Data Index: 1820686 (The Pile)

Max Activating Token Index: 235

Text #2

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.8831. Min Act: -0.0000

Data Index: 1759899 (The Pile)

Max Activating Token Index: 500

Text #3

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.6680. Min Act: -0.0000

Data Index: 1905524 (The Pile)

Max Activating Token Index: 226

Text #4

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.5649. Min Act: -0.0000

Data Index: 516171 (The Pile)

Max Activating Token Index: 626

Text #5

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4825. Min Act: -0.0000

Data Index: 426975 (The Pile)

Max Activating Token Index: 553

Text #6

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4803. Min Act: -0.0000

Data Index: 1835462 (The Pile)

Max Activating Token Index: 568

Text #7

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.2813. Min Act: -0.0000

Data Index: 1580554 (The Pile)

Max Activating Token Index: 750

Text #8

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4175. Min Act: -0.0000

Data Index: 403894 (The Pile)

Max Activating Token Index: 739

Text #9

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3952. Min Act: -0.0000

Data Index: 1011843 (The Pile)

Max Activating Token Index: 574

Text #10

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3781. Min Act: -0.0000

Data Index: 1844964 (The Pile)

Max Activating Token Index: 294

Text #11

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4580. Min Act: -0.0000

Data Index: 88035 (The Pile)

Max Activating Token Index: 137

Text #12

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3047. Min Act: -0.0000

Data Index: 998898 (The Pile)

Max Activating Token Index: 547

Text #13

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.2987. Min Act: -0.0000

Data Index: 111773 (The Pile)

Max Activating Token Index: 807

Text #14

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3667. Min Act: -0.0000

Data Index: 773057 (The Pile)

Max Activating Token Index: 459

Text #15

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4338. Min Act: -0.0000

Data Index: 659520 (The Pile)

Max Activating Token Index: 33

Text #16

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3405. Min Act: -0.0000

Data Index: 1677414 (The Pile)

Max Activating Token Index: 695

Text #17

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.5080. Min Act: -0.0000

Data Index: 1825800 (The Pile)

Max Activating Token Index: 1019

Text #18

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.3409. Min Act: -0.0000

Data Index: 1408118 (The Pile)

Max Activating Token Index: 351

Text #19

Max Range: 1.8831. Min Range: -1.8831

Max Act: 1.4050. Min Act: -0.0000

Data Index: 1187291 (The Pile)

Max Activating Token Index: 700