Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4077 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.5082. Min Act: -0.0000

Data Index: 426748 (The Pile)

Max Activating Token Index: 718

Text #1

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4876. Min Act: -0.0000

Data Index: 1774035 (The Pile)

Max Activating Token Index: 579

Text #2

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4596. Min Act: -0.0000

Data Index: 1246798 (The Pile)

Max Activating Token Index: 648

Text #3

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4834. Min Act: -0.0000

Data Index: 708655 (The Pile)

Max Activating Token Index: 222

Text #4

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4367. Min Act: -0.0000

Data Index: 239183 (The Pile)

Max Activating Token Index: 224

Text #5

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4538. Min Act: -0.0000

Data Index: 1879265 (The Pile)

Max Activating Token Index: 915

Text #6

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.4060. Min Act: -0.0000

Data Index: 1079104 (The Pile)

Max Activating Token Index: 101

Text #7

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3298. Min Act: -0.0000

Data Index: 798698 (The Pile)

Max Activating Token Index: 179

Text #8

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3363. Min Act: -0.0000

Data Index: 504282 (The Pile)

Max Activating Token Index: 113

Text #9

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3489. Min Act: -0.0000

Data Index: 562994 (The Pile)

Max Activating Token Index: 170

Text #10

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3228. Min Act: -0.0000

Data Index: 945183 (The Pile)

Max Activating Token Index: 130

Text #11

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3089. Min Act: -0.0000

Data Index: 28344 (The Pile)

Max Activating Token Index: 302

Text #12

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2953. Min Act: -0.0000

Data Index: 677578 (The Pile)

Max Activating Token Index: 889

Text #13

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2507. Min Act: -0.0000

Data Index: 1544163 (The Pile)

Max Activating Token Index: 762

Text #14

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.3050. Min Act: -0.0000

Data Index: 692207 (The Pile)

Max Activating Token Index: 606

Text #15

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2850. Min Act: -0.0000

Data Index: 1284959 (The Pile)

Max Activating Token Index: 635

Text #16

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2839. Min Act: -0.0000

Data Index: 1886661 (The Pile)

Max Activating Token Index: 761

Text #17

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2928. Min Act: -0.0000

Data Index: 1513605 (The Pile)

Max Activating Token Index: 204

Text #18

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2983. Min Act: -0.0000

Data Index: 1789786 (The Pile)

Max Activating Token Index: 857

Text #19

Max Range: 0.5082. Min Range: -0.5082

Max Act: 0.2892. Min Act: -0.0000

Data Index: 1118933 (The Pile)

Max Activating Token Index: 507