Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2574 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.6372. Min Act: -0.0000

Data Index: 424636 (The Pile)

Max Activating Token Index: 898

Text #1

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5650. Min Act: -0.0000

Data Index: 1651546 (The Pile)

Max Activating Token Index: 80

Text #2

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5952. Min Act: -0.0000

Data Index: 514332 (The Pile)

Max Activating Token Index: 902

Text #3

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5173. Min Act: -0.0000

Data Index: 1754945 (The Pile)

Max Activating Token Index: 616

Text #4

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5422. Min Act: -0.0000

Data Index: 760826 (The Pile)

Max Activating Token Index: 793

Text #5

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5768. Min Act: -0.0000

Data Index: 233245 (The Pile)

Max Activating Token Index: 696

Text #6

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5525. Min Act: -0.0000

Data Index: 1200932 (The Pile)

Max Activating Token Index: 387

Text #7

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5656. Min Act: -0.0000

Data Index: 262721 (The Pile)

Max Activating Token Index: 9

Text #8

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5289. Min Act: -0.0000

Data Index: 1408999 (The Pile)

Max Activating Token Index: 556

Text #9

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5612. Min Act: -0.0000

Data Index: 1793199 (The Pile)

Max Activating Token Index: 893

Text #10

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5162. Min Act: -0.0000

Data Index: 564241 (The Pile)

Max Activating Token Index: 974

Text #11

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5707. Min Act: -0.0000

Data Index: 224313 (The Pile)

Max Activating Token Index: 806

Text #12

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5487. Min Act: -0.0000

Data Index: 701013 (The Pile)

Max Activating Token Index: 490

Text #13

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5830. Min Act: -0.0000

Data Index: 335983 (The Pile)

Max Activating Token Index: 672

Text #14

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5309. Min Act: -0.0000

Data Index: 846731 (The Pile)

Max Activating Token Index: 338

Text #15

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5451. Min Act: -0.0000

Data Index: 1565294 (The Pile)

Max Activating Token Index: 446

Text #16

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5271. Min Act: -0.0000

Data Index: 1966265 (The Pile)

Max Activating Token Index: 687

Text #17

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5250. Min Act: -0.0000

Data Index: 1629983 (The Pile)

Max Activating Token Index: 529

Text #18

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5874. Min Act: -0.0000

Data Index: 1354861 (The Pile)

Max Activating Token Index: 980

Text #19

Max Range: 0.6372. Min Range: -0.6372

Max Act: 0.5414. Min Act: -0.0000

Data Index: 1767871 (The Pile)

Max Activating Token Index: 787