Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1329 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5708. Min Act: -0.0000

Data Index: 98212 (The Pile)

Max Activating Token Index: 314

Text #1

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.6120. Min Act: -0.0000

Data Index: 552402 (The Pile)

Max Activating Token Index: 472

Text #2

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5977. Min Act: -0.0000

Data Index: 1807163 (The Pile)

Max Activating Token Index: 386

Text #3

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5729. Min Act: -0.0000

Data Index: 1979295 (The Pile)

Max Activating Token Index: 357

Text #4

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5810. Min Act: -0.0000

Data Index: 1144890 (The Pile)

Max Activating Token Index: 473

Text #5

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5353. Min Act: -0.0000

Data Index: 474194 (The Pile)

Max Activating Token Index: 162

Text #6

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5373. Min Act: -0.0000

Data Index: 198926 (The Pile)

Max Activating Token Index: 1023

Text #7

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4847. Min Act: -0.0000

Data Index: 462685 (The Pile)

Max Activating Token Index: 618

Text #8

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4697. Min Act: -0.0000

Data Index: 223333 (The Pile)

Max Activating Token Index: 842

Text #9

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.7683. Min Act: -0.0000

Data Index: 997084 (The Pile)

Max Activating Token Index: 940

Text #10

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.3934. Min Act: -0.0001

Data Index: 607551 (The Pile)

Max Activating Token Index: 986

Text #11

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4626. Min Act: -0.0000

Data Index: 441893 (The Pile)

Max Activating Token Index: 471

Text #12

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4871. Min Act: -0.0000

Data Index: 516693 (The Pile)

Max Activating Token Index: 856

Text #13

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5071. Min Act: -0.0000

Data Index: 388250 (The Pile)

Max Activating Token Index: 829

Text #14

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.3070. Min Act: -0.0000

Data Index: 1415481 (The Pile)

Max Activating Token Index: 981

Text #15

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5256. Min Act: -0.0000

Data Index: 143578 (The Pile)

Max Activating Token Index: 319

Text #16

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4286. Min Act: -0.0000

Data Index: 168890 (The Pile)

Max Activating Token Index: 830

Text #17

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.5275. Min Act: -0.0000

Data Index: 541228 (The Pile)

Max Activating Token Index: 917

Text #18

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4176. Min Act: -0.0000

Data Index: 406284 (The Pile)

Max Activating Token Index: 686

Text #19

Max Range: 1.7683. Min Range: -1.7683

Max Act: 1.4142. Min Act: -0.0000

Data Index: 848680 (The Pile)

Max Activating Token Index: 394