Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 588 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4457. Min Act: -0.0000

Data Index: 555942 (The Pile)

Max Activating Token Index: 345

Text #1

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4599. Min Act: -0.0000

Data Index: 1436080 (The Pile)

Max Activating Token Index: 380

Text #2

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4294. Min Act: -0.0000

Data Index: 630660 (The Pile)

Max Activating Token Index: 630

Text #3

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4127. Min Act: -0.0000

Data Index: 760780 (The Pile)

Max Activating Token Index: 139

Text #4

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4303. Min Act: -0.0000

Data Index: 1793841 (The Pile)

Max Activating Token Index: 848

Text #5

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4122. Min Act: -0.0000

Data Index: 216485 (The Pile)

Max Activating Token Index: 106

Text #6

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4135. Min Act: -0.0000

Data Index: 1475218 (The Pile)

Max Activating Token Index: 229

Text #7

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4288. Min Act: -0.0000

Data Index: 1591558 (The Pile)

Max Activating Token Index: 415

Text #8

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4232. Min Act: -0.0000

Data Index: 196301 (The Pile)

Max Activating Token Index: 112

Text #9

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4002. Min Act: -0.0000

Data Index: 1130317 (The Pile)

Max Activating Token Index: 1023

Text #10

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3845. Min Act: -0.0000

Data Index: 783090 (The Pile)

Max Activating Token Index: 657

Text #11

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4448. Min Act: -0.0000

Data Index: 76756 (The Pile)

Max Activating Token Index: 585

Text #12

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3902. Min Act: -0.0000

Data Index: 1927026 (The Pile)

Max Activating Token Index: 298

Text #13

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3696. Min Act: -0.0000

Data Index: 1760337 (The Pile)

Max Activating Token Index: 609

Text #14

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4213. Min Act: -0.0000

Data Index: 112221 (The Pile)

Max Activating Token Index: 339

Text #15

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3590. Min Act: -0.0000

Data Index: 1803393 (The Pile)

Max Activating Token Index: 1023

Text #16

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3992. Min Act: -0.0000

Data Index: 134784 (The Pile)

Max Activating Token Index: 29

Text #17

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.4125. Min Act: -0.0000

Data Index: 1109922 (The Pile)

Max Activating Token Index: 478

Text #18

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3980. Min Act: -0.0000

Data Index: 1863275 (The Pile)

Max Activating Token Index: 141

Text #19

Max Range: 0.4599. Min Range: -0.4599

Max Act: 0.3817. Min Act: -0.0000

Data Index: 248338 (The Pile)

Max Activating Token Index: 176