Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 846 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.7083. Min Act: -0.0000

Data Index: 258468 (The Pile)

Max Activating Token Index: 387

Text #1

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6431. Min Act: -0.0000

Data Index: 919733 (The Pile)

Max Activating Token Index: 6

Text #2

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6828. Min Act: -0.0000

Data Index: 876494 (The Pile)

Max Activating Token Index: 11

Text #3

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6733. Min Act: -0.0000

Data Index: 1554945 (The Pile)

Max Activating Token Index: 412

Text #4

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6039. Min Act: -0.0000

Data Index: 629549 (The Pile)

Max Activating Token Index: 770

Text #5

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6541. Min Act: -0.0000

Data Index: 1136364 (The Pile)

Max Activating Token Index: 567

Text #6

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5691. Min Act: -0.0000

Data Index: 261584 (The Pile)

Max Activating Token Index: 59

Text #7

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5063. Min Act: -0.0000

Data Index: 1671609 (The Pile)

Max Activating Token Index: 679

Text #8

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6620. Min Act: -0.0000

Data Index: 78731 (The Pile)

Max Activating Token Index: 55

Text #9

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6078. Min Act: -0.0000

Data Index: 778952 (The Pile)

Max Activating Token Index: 806

Text #10

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6308. Min Act: -0.0000

Data Index: 1932523 (The Pile)

Max Activating Token Index: 266

Text #11

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5299. Min Act: -0.0000

Data Index: 432574 (The Pile)

Max Activating Token Index: 42

Text #12

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5204. Min Act: -0.0000

Data Index: 1563764 (The Pile)

Max Activating Token Index: 468

Text #13

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6339. Min Act: -0.0000

Data Index: 853583 (The Pile)

Max Activating Token Index: 180

Text #14

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5363. Min Act: -0.0000

Data Index: 280272 (The Pile)

Max Activating Token Index: 641

Text #15

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5083. Min Act: -0.0000

Data Index: 1629379 (The Pile)

Max Activating Token Index: 688

Text #16

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6139. Min Act: -0.0000

Data Index: 121168 (The Pile)

Max Activating Token Index: 577

Text #17

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5858. Min Act: -0.0000

Data Index: 729670 (The Pile)

Max Activating Token Index: 7

Text #18

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.6198. Min Act: -0.0000

Data Index: 70164 (The Pile)

Max Activating Token Index: 11

Text #19

Max Range: 0.7083. Min Range: -0.7083

Max Act: 0.5935. Min Act: -0.0000

Data Index: 1254277 (The Pile)

Max Activating Token Index: 634