Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4349 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.8106. Min Act: -0.0000

Data Index: 1114660 (The Pile)

Max Activating Token Index: 749

Text #1

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.7060. Min Act: -0.0000

Data Index: 944048 (The Pile)

Max Activating Token Index: 833

Text #2

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.6546. Min Act: -0.0000

Data Index: 1249070 (The Pile)

Max Activating Token Index: 559

Text #3

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.6690. Min Act: -0.0000

Data Index: 1280102 (The Pile)

Max Activating Token Index: 706

Text #4

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.5808. Min Act: -0.0000

Data Index: 1415605 (The Pile)

Max Activating Token Index: 448

Text #5

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.6308. Min Act: -0.0000

Data Index: 1300097 (The Pile)

Max Activating Token Index: 541

Text #6

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4664. Min Act: -0.0000

Data Index: 290741 (The Pile)

Max Activating Token Index: 121

Text #7

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.6538. Min Act: -0.0000

Data Index: 1193945 (The Pile)

Max Activating Token Index: 618

Text #8

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.5749. Min Act: -0.0000

Data Index: 1882444 (The Pile)

Max Activating Token Index: 87

Text #9

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4559. Min Act: -0.0000

Data Index: 1517103 (The Pile)

Max Activating Token Index: 499

Text #10

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4009. Min Act: -0.0000

Data Index: 1766623 (The Pile)

Max Activating Token Index: 975

Text #11

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4023. Min Act: -0.0000

Data Index: 571867 (The Pile)

Max Activating Token Index: 377

Text #12

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4843. Min Act: -0.0000

Data Index: 328140 (The Pile)

Max Activating Token Index: 168

Text #13

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4858. Min Act: -0.0000

Data Index: 964094 (The Pile)

Max Activating Token Index: 576

Text #14

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4196. Min Act: -0.0000

Data Index: 1327357 (The Pile)

Max Activating Token Index: 195

Text #15

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4277. Min Act: -0.0000

Data Index: 843409 (The Pile)

Max Activating Token Index: 681

Text #16

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.4076. Min Act: -0.0000

Data Index: 1530761 (The Pile)

Max Activating Token Index: 154

Text #17

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.3801. Min Act: -0.0000

Data Index: 1428512 (The Pile)

Max Activating Token Index: 455

Text #18

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.3813. Min Act: -0.0000

Data Index: 1634908 (The Pile)

Max Activating Token Index: 269

Text #19

Max Range: 0.8106. Min Range: -0.8106

Max Act: 0.3980. Min Act: -0.0000

Data Index: 1874130 (The Pile)

Max Activating Token Index: 62