Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2230 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.0146. Min Range: -5.0146

Max Act: 5.0146. Min Act: -0.0000

Data Index: 16720 (The Pile)

Max Activating Token Index: 900

Text #1

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.8759. Min Act: -0.0000

Data Index: 1319863 (The Pile)

Max Activating Token Index: 688

Text #2

Max Range: 5.0146. Min Range: -5.0146

Max Act: 4.5034. Min Act: -0.0000

Data Index: 898474 (The Pile)

Max Activating Token Index: 411

Text #3

Max Range: 5.0146. Min Range: -5.0146

Max Act: 4.0386. Min Act: -0.0000

Data Index: 1168782 (The Pile)

Max Activating Token Index: 352

Text #4

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.8194. Min Act: -0.0000

Data Index: 1106865 (The Pile)

Max Activating Token Index: 535

Text #5

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.8933. Min Act: -0.0000

Data Index: 48797 (The Pile)

Max Activating Token Index: 327

Text #6

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.9456. Min Act: 0.0000

Data Index: 611546 (The Pile)

Max Activating Token Index: 900

Text #7

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.7058. Min Act: 0.0000

Data Index: 661107 (The Pile)

Max Activating Token Index: 465

Text #8

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.9634. Min Act: -0.0000

Data Index: 1133337 (The Pile)

Max Activating Token Index: 576

Text #9

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.4890. Min Act: 0.0000

Data Index: 475703 (The Pile)

Max Activating Token Index: 790

Text #10

Max Range: 5.0146. Min Range: -5.0146

Max Act: 4.0207. Min Act: -0.0000

Data Index: 934813 (The Pile)

Max Activating Token Index: 409

Text #11

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.5649. Min Act: 0.0000

Data Index: 1067773 (The Pile)

Max Activating Token Index: 698

Text #12

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.5235. Min Act: -0.0000

Data Index: 1521253 (The Pile)

Max Activating Token Index: 836

Text #13

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.6808. Min Act: 0.0000

Data Index: 982126 (The Pile)

Max Activating Token Index: 777

Text #14

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.6802. Min Act: 0.0000

Data Index: 1272184 (The Pile)

Max Activating Token Index: 836

Text #15

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.6654. Min Act: -0.0000

Data Index: 1947843 (The Pile)

Max Activating Token Index: 477

Text #16

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.2782. Min Act: -0.0000

Data Index: 574564 (The Pile)

Max Activating Token Index: 653

Text #17

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.2293. Min Act: -0.0000

Data Index: 367111 (The Pile)

Max Activating Token Index: 634

Text #18

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.5380. Min Act: -0.0000

Data Index: 1308376 (The Pile)

Max Activating Token Index: 1009

Text #19

Max Range: 5.0146. Min Range: -5.0146

Max Act: 3.1383. Min Act: 0.0000

Data Index: 261125 (The Pile)

Max Activating Token Index: 225