Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1656 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2838. Min Act: -0.0000

Data Index: 1124983 (The Pile)

Max Activating Token Index: 420

Text #1

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2755. Min Act: -0.0000

Data Index: 1303311 (The Pile)

Max Activating Token Index: 378

Text #2

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2101. Min Act: -0.0000

Data Index: 1548665 (The Pile)

Max Activating Token Index: 562

Text #3

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2088. Min Act: -0.0000

Data Index: 1639386 (The Pile)

Max Activating Token Index: 348

Text #4

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2131. Min Act: -0.0000

Data Index: 1405273 (The Pile)

Max Activating Token Index: 325

Text #5

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2197. Min Act: -0.0000

Data Index: 358846 (The Pile)

Max Activating Token Index: 569

Text #6

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2091. Min Act: -0.0000

Data Index: 756250 (The Pile)

Max Activating Token Index: 352

Text #7

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2081. Min Act: -0.0000

Data Index: 1239645 (The Pile)

Max Activating Token Index: 428

Text #8

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2011. Min Act: -0.0000

Data Index: 995704 (The Pile)

Max Activating Token Index: 532

Text #9

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2072. Min Act: -0.0000

Data Index: 1916434 (The Pile)

Max Activating Token Index: 321

Text #10

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2052. Min Act: -0.0000

Data Index: 548079 (The Pile)

Max Activating Token Index: 411

Text #11

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1972. Min Act: -0.0000

Data Index: 311654 (The Pile)

Max Activating Token Index: 615

Text #12

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1918. Min Act: -0.0000

Data Index: 498585 (The Pile)

Max Activating Token Index: 59

Text #13

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1956. Min Act: -0.0000

Data Index: 476484 (The Pile)

Max Activating Token Index: 321

Text #14

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2025. Min Act: -0.0000

Data Index: 1621279 (The Pile)

Max Activating Token Index: 293

Text #15

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2002. Min Act: -0.0000

Data Index: 46870 (The Pile)

Max Activating Token Index: 518

Text #16

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1933. Min Act: -0.0000

Data Index: 1582909 (The Pile)

Max Activating Token Index: 308

Text #17

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1874. Min Act: -0.0000

Data Index: 899017 (The Pile)

Max Activating Token Index: 127

Text #18

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.1888. Min Act: -0.0000

Data Index: 55419 (The Pile)

Max Activating Token Index: 670

Text #19

Max Range: 0.2838. Min Range: -0.2838

Max Act: 0.2001. Min Act: -0.0000

Data Index: 229400 (The Pile)

Max Activating Token Index: 611