Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 394 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5913. Min Act: -0.0000

Data Index: 1304653 (The Pile)

Max Activating Token Index: 557

Text #1

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.6535. Min Act: -0.0000

Data Index: 357921 (The Pile)

Max Activating Token Index: 626

Text #2

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5674. Min Act: -0.0000

Data Index: 1248524 (The Pile)

Max Activating Token Index: 363

Text #3

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5583. Min Act: -0.0000

Data Index: 683703 (The Pile)

Max Activating Token Index: 945

Text #4

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5437. Min Act: -0.0000

Data Index: 387874 (The Pile)

Max Activating Token Index: 483

Text #5

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5432. Min Act: -0.0000

Data Index: 1362435 (The Pile)

Max Activating Token Index: 628

Text #6

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5021. Min Act: -0.0000

Data Index: 917943 (The Pile)

Max Activating Token Index: 851

Text #7

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5158. Min Act: -0.0000

Data Index: 263172 (The Pile)

Max Activating Token Index: 992

Text #8

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5023. Min Act: -0.0000

Data Index: 1389505 (The Pile)

Max Activating Token Index: 587

Text #9

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4870. Min Act: -0.0000

Data Index: 795710 (The Pile)

Max Activating Token Index: 559

Text #10

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5199. Min Act: -0.0000

Data Index: 924380 (The Pile)

Max Activating Token Index: 112

Text #11

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4526. Min Act: -0.0000

Data Index: 593122 (The Pile)

Max Activating Token Index: 23

Text #12

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5152. Min Act: -0.0000

Data Index: 240008 (The Pile)

Max Activating Token Index: 242

Text #13

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4791. Min Act: -0.0000

Data Index: 1545394 (The Pile)

Max Activating Token Index: 138

Text #14

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4765. Min Act: -0.0000

Data Index: 1841740 (The Pile)

Max Activating Token Index: 244

Text #15

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5081. Min Act: -0.0000

Data Index: 1536226 (The Pile)

Max Activating Token Index: 785

Text #16

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.5022. Min Act: -0.0000

Data Index: 1611141 (The Pile)

Max Activating Token Index: 219

Text #17

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4887. Min Act: -0.0000

Data Index: 533622 (The Pile)

Max Activating Token Index: 81

Text #18

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4858. Min Act: -0.0000

Data Index: 292523 (The Pile)

Max Activating Token Index: 580

Text #19

Max Range: 0.6535. Min Range: -0.6535

Max Act: 0.4827. Min Act: -0.0000

Data Index: 1714793 (The Pile)

Max Activating Token Index: 172