Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2117 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.8294. Min Act: -0.0000

Data Index: 1529313 (The Pile)

Max Activating Token Index: 1001

Text #1

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.6933. Min Act: -0.0000

Data Index: 338155 (The Pile)

Max Activating Token Index: 844

Text #2

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.7266. Min Act: -0.0000

Data Index: 1957018 (The Pile)

Max Activating Token Index: 970

Text #3

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5696. Min Act: -0.0000

Data Index: 46948 (The Pile)

Max Activating Token Index: 157

Text #4

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.6461. Min Act: -0.0000

Data Index: 288432 (The Pile)

Max Activating Token Index: 349

Text #5

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5193. Min Act: -0.0000

Data Index: 1069193 (The Pile)

Max Activating Token Index: 923

Text #6

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4334. Min Act: -0.0000

Data Index: 348824 (The Pile)

Max Activating Token Index: 267

Text #7

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.6122. Min Act: -0.0000

Data Index: 693125 (The Pile)

Max Activating Token Index: 437

Text #8

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4334. Min Act: -0.0000

Data Index: 1307708 (The Pile)

Max Activating Token Index: 407

Text #9

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5891. Min Act: -0.0000

Data Index: 15016 (The Pile)

Max Activating Token Index: 375

Text #10

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5063. Min Act: -0.0000

Data Index: 1212797 (The Pile)

Max Activating Token Index: 597

Text #11

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.6610. Min Act: -0.0000

Data Index: 847276 (The Pile)

Max Activating Token Index: 101

Text #12

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5790. Min Act: -0.0000

Data Index: 1344860 (The Pile)

Max Activating Token Index: 499

Text #13

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4848. Min Act: -0.0000

Data Index: 1700700 (The Pile)

Max Activating Token Index: 588

Text #14

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.3834. Min Act: -0.0000

Data Index: 844058 (The Pile)

Max Activating Token Index: 35

Text #15

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.6122. Min Act: -0.0000

Data Index: 674493 (The Pile)

Max Activating Token Index: 803

Text #16

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4612. Min Act: -0.0000

Data Index: 1949919 (The Pile)

Max Activating Token Index: 1001

Text #17

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4635. Min Act: -0.0000

Data Index: 189177 (The Pile)

Max Activating Token Index: 985

Text #18

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.4310. Min Act: -0.0000

Data Index: 213122 (The Pile)

Max Activating Token Index: 256

Text #19

Max Range: 1.8294. Min Range: -1.8294

Max Act: 1.5290. Min Act: -0.0000

Data Index: 694611 (The Pile)

Max Activating Token Index: 512