Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4877 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.9997. Min Act: -0.0000

Data Index: 853542 (The Pile)

Max Activating Token Index: 968

Text #1

Max Range: 1.0685. Min Range: -1.0685

Max Act: 1.0436. Min Act: -0.0000

Data Index: 1823591 (The Pile)

Max Activating Token Index: 399

Text #2

Max Range: 1.0685. Min Range: -1.0685

Max Act: 1.0685. Min Act: -0.0000

Data Index: 1156690 (The Pile)

Max Activating Token Index: 104

Text #3

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.9620. Min Act: -0.0000

Data Index: 1119673 (The Pile)

Max Activating Token Index: 463

Text #4

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.9790. Min Act: -0.0000

Data Index: 1975553 (The Pile)

Max Activating Token Index: 217

Text #5

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.9593. Min Act: -0.0000

Data Index: 898982 (The Pile)

Max Activating Token Index: 306

Text #6

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8314. Min Act: -0.0001

Data Index: 828718 (The Pile)

Max Activating Token Index: 591

Text #7

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8833. Min Act: -0.0000

Data Index: 897316 (The Pile)

Max Activating Token Index: 507

Text #8

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.7495. Min Act: -0.0000

Data Index: 888020 (The Pile)

Max Activating Token Index: 826

Text #9

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8512. Min Act: -0.0000

Data Index: 976215 (The Pile)

Max Activating Token Index: 303

Text #10

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.7231. Min Act: -0.0000

Data Index: 1061009 (The Pile)

Max Activating Token Index: 306

Text #11

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8056. Min Act: -0.0000

Data Index: 1331068 (The Pile)

Max Activating Token Index: 566

Text #12

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8108. Min Act: -0.0000

Data Index: 1506002 (The Pile)

Max Activating Token Index: 362

Text #13

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.7559. Min Act: -0.0000

Data Index: 1310824 (The Pile)

Max Activating Token Index: 153

Text #14

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8707. Min Act: -0.0000

Data Index: 303582 (The Pile)

Max Activating Token Index: 598

Text #15

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8210. Min Act: -0.0000

Data Index: 81096 (The Pile)

Max Activating Token Index: 867

Text #16

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8056. Min Act: -0.0000

Data Index: 488157 (The Pile)

Max Activating Token Index: 399

Text #17

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.7891. Min Act: -0.0000

Data Index: 268437 (The Pile)

Max Activating Token Index: 563

Text #18

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.8053. Min Act: -0.0000

Data Index: 1788340 (The Pile)

Max Activating Token Index: 818

Text #19

Max Range: 1.0685. Min Range: -1.0685

Max Act: 0.7199. Min Act: -0.0000

Data Index: 64383 (The Pile)

Max Activating Token Index: 1017