Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4606 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.7754. Min Act: -0.0000

Data Index: 952219 (The Pile)

Max Activating Token Index: 786

Text #1

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.8052. Min Act: -0.0000

Data Index: 1068751 (The Pile)

Max Activating Token Index: 562

Text #2

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.8383. Min Act: -0.0000

Data Index: 1700439 (The Pile)

Max Activating Token Index: 426

Text #3

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.7284. Min Act: -0.0000

Data Index: 851125 (The Pile)

Max Activating Token Index: 585

Text #4

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5889. Min Act: -0.0000

Data Index: 165487 (The Pile)

Max Activating Token Index: 15

Text #5

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.6587. Min Act: -0.0000

Data Index: 1226022 (The Pile)

Max Activating Token Index: 472

Text #6

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.7648. Min Act: -0.0000

Data Index: 268177 (The Pile)

Max Activating Token Index: 665

Text #7

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5748. Min Act: -0.0000

Data Index: 1817147 (The Pile)

Max Activating Token Index: 441

Text #8

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.6035. Min Act: -0.0000

Data Index: 1749787 (The Pile)

Max Activating Token Index: 874

Text #9

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5124. Min Act: -0.0000

Data Index: 371134 (The Pile)

Max Activating Token Index: 848

Text #10

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5887. Min Act: -0.0000

Data Index: 602681 (The Pile)

Max Activating Token Index: 200

Text #11

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.7579. Min Act: -0.0000

Data Index: 252464 (The Pile)

Max Activating Token Index: 315

Text #12

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.4697. Min Act: -0.0000

Data Index: 953469 (The Pile)

Max Activating Token Index: 19

Text #13

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.6288. Min Act: -0.0000

Data Index: 216436 (The Pile)

Max Activating Token Index: 564

Text #14

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5360. Min Act: -0.0000

Data Index: 324124 (The Pile)

Max Activating Token Index: 499

Text #15

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5320. Min Act: -0.0000

Data Index: 1631347 (The Pile)

Max Activating Token Index: 629

Text #16

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.6041. Min Act: -0.0000

Data Index: 1290320 (The Pile)

Max Activating Token Index: 515

Text #17

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.6170. Min Act: -0.0000

Data Index: 1760627 (The Pile)

Max Activating Token Index: 1013

Text #18

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5166. Min Act: -0.0000

Data Index: 1452754 (The Pile)

Max Activating Token Index: 449

Text #19

Max Range: 1.8383. Min Range: -1.8383

Max Act: 1.5127. Min Act: -0.0000

Data Index: 552981 (The Pile)

Max Activating Token Index: 29