Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1588 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0193. Min Range: -1.0193

Max Act: 1.0193. Min Act: -0.0000

Data Index: 753382 (The Pile)

Max Activating Token Index: 30

Text #1

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.8184. Min Act: -0.0000

Data Index: 1396497 (The Pile)

Max Activating Token Index: 984

Text #2

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.8577. Min Act: -0.0000

Data Index: 733024 (The Pile)

Max Activating Token Index: 895

Text #3

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6569. Min Act: -0.0000

Data Index: 617517 (The Pile)

Max Activating Token Index: 843

Text #4

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.7232. Min Act: -0.0000

Data Index: 459796 (The Pile)

Max Activating Token Index: 1004

Text #5

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6733. Min Act: -0.0000

Data Index: 242227 (The Pile)

Max Activating Token Index: 164

Text #6

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6104. Min Act: -0.0000

Data Index: 1461152 (The Pile)

Max Activating Token Index: 795

Text #7

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6792. Min Act: -0.0000

Data Index: 1809327 (The Pile)

Max Activating Token Index: 815

Text #8

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.7037. Min Act: -0.0000

Data Index: 1667629 (The Pile)

Max Activating Token Index: 617

Text #9

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6532. Min Act: -0.0000

Data Index: 744543 (The Pile)

Max Activating Token Index: 544

Text #10

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6234. Min Act: -0.0000

Data Index: 870036 (The Pile)

Max Activating Token Index: 576

Text #11

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.7619. Min Act: -0.0000

Data Index: 968939 (The Pile)

Max Activating Token Index: 962

Text #12

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.7011. Min Act: -0.0000

Data Index: 597299 (The Pile)

Max Activating Token Index: 696

Text #13

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6487. Min Act: -0.0000

Data Index: 949381 (The Pile)

Max Activating Token Index: 308

Text #14

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6582. Min Act: -0.0000

Data Index: 669880 (The Pile)

Max Activating Token Index: 990

Text #15

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.5632. Min Act: -0.0000

Data Index: 1963349 (The Pile)

Max Activating Token Index: 760

Text #16

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6776. Min Act: -0.0000

Data Index: 603127 (The Pile)

Max Activating Token Index: 144

Text #17

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.5882. Min Act: -0.0000

Data Index: 774831 (The Pile)

Max Activating Token Index: 1010

Text #18

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.6325. Min Act: -0.0000

Data Index: 905067 (The Pile)

Max Activating Token Index: 280

Text #19

Max Range: 1.0193. Min Range: -1.0193

Max Act: 0.5779. Min Act: -0.0000

Data Index: 290554 (The Pile)

Max Activating Token Index: 188