Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2522 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.9377. Min Range: -5.9377

Max Act: 5.9377. Min Act: -0.0000

Data Index: 1327139 (The Pile)

Max Activating Token Index: 592

Text #1

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.5186. Min Act: -0.0000

Data Index: 726115 (The Pile)

Max Activating Token Index: 911

Text #2

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.7465. Min Act: -0.0000

Data Index: 1246935 (The Pile)

Max Activating Token Index: 256

Text #3

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.7511. Min Act: -0.0000

Data Index: 419598 (The Pile)

Max Activating Token Index: 932

Text #4

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.6653. Min Act: -0.0000

Data Index: 639070 (The Pile)

Max Activating Token Index: 872

Text #5

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.1614. Min Act: -0.0000

Data Index: 1907155 (The Pile)

Max Activating Token Index: 523

Text #6

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.2547. Min Act: -0.0000

Data Index: 1521455 (The Pile)

Max Activating Token Index: 139

Text #7

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.1328. Min Act: -0.0000

Data Index: 469432 (The Pile)

Max Activating Token Index: 595

Text #8

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.1061. Min Act: -0.0000

Data Index: 856599 (The Pile)

Max Activating Token Index: 751

Text #9

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.9430. Min Act: -0.0000

Data Index: 429736 (The Pile)

Max Activating Token Index: 626

Text #10

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.6974. Min Act: -0.0000

Data Index: 48554 (The Pile)

Max Activating Token Index: 604

Text #11

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.7339. Min Act: -0.0000

Data Index: 248674 (The Pile)

Max Activating Token Index: 327

Text #12

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.5723. Min Act: -0.0000

Data Index: 402962 (The Pile)

Max Activating Token Index: 808

Text #13

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.7113. Min Act: -0.0000

Data Index: 746798 (The Pile)

Max Activating Token Index: 124

Text #14

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.7393. Min Act: -0.0000

Data Index: 1694506 (The Pile)

Max Activating Token Index: 970

Text #15

Max Range: 5.9377. Min Range: -5.9377

Max Act: 4.1783. Min Act: -0.0000

Data Index: 1198497 (The Pile)

Max Activating Token Index: 812

Text #16

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.6434. Min Act: -0.0000

Data Index: 456241 (The Pile)

Max Activating Token Index: 796

Text #17

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.6718. Min Act: -0.0000

Data Index: 1905581 (The Pile)

Max Activating Token Index: 604

Text #18

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.2357. Min Act: -0.0000

Data Index: 319094 (The Pile)

Max Activating Token Index: 706

Text #19

Max Range: 5.9377. Min Range: -5.9377

Max Act: 3.7291. Min Act: -0.0000

Data Index: 430594 (The Pile)

Max Activating Token Index: 126