Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2351 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1592. Min Range: -1.1592

Max Act: 1.1592. Min Act: -0.0000

Data Index: 1080338 (The Pile)

Max Activating Token Index: 927

Text #1

Max Range: 1.1592. Min Range: -1.1592

Max Act: 1.1561. Min Act: -0.0000

Data Index: 160247 (The Pile)

Max Activating Token Index: 295

Text #2

Max Range: 1.1592. Min Range: -1.1592

Max Act: 1.0509. Min Act: -0.0000

Data Index: 1512803 (The Pile)

Max Activating Token Index: 545

Text #3

Max Range: 1.1592. Min Range: -1.1592

Max Act: 1.0019. Min Act: -0.0000

Data Index: 1315966 (The Pile)

Max Activating Token Index: 126

Text #4

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.9124. Min Act: -0.0000

Data Index: 669858 (The Pile)

Max Activating Token Index: 891

Text #5

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8823. Min Act: -0.0000

Data Index: 1879190 (The Pile)

Max Activating Token Index: 16

Text #6

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.9325. Min Act: -0.0000

Data Index: 703945 (The Pile)

Max Activating Token Index: 637

Text #7

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8380. Min Act: -0.0000

Data Index: 289002 (The Pile)

Max Activating Token Index: 815

Text #8

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8791. Min Act: -0.0000

Data Index: 1377518 (The Pile)

Max Activating Token Index: 430

Text #9

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8100. Min Act: -0.0000

Data Index: 1642197 (The Pile)

Max Activating Token Index: 746

Text #10

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8678. Min Act: -0.0000

Data Index: 541635 (The Pile)

Max Activating Token Index: 150

Text #11

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8737. Min Act: -0.0000

Data Index: 281427 (The Pile)

Max Activating Token Index: 187

Text #12

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.7690. Min Act: -0.0000

Data Index: 889467 (The Pile)

Max Activating Token Index: 805

Text #13

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8223. Min Act: -0.0000

Data Index: 1262934 (The Pile)

Max Activating Token Index: 515

Text #14

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8542. Min Act: -0.0000

Data Index: 402503 (The Pile)

Max Activating Token Index: 1009

Text #15

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8338. Min Act: -0.0000

Data Index: 1668990 (The Pile)

Max Activating Token Index: 414

Text #16

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8021. Min Act: -0.0000

Data Index: 720744 (The Pile)

Max Activating Token Index: 289

Text #17

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.8372. Min Act: -0.0000

Data Index: 927633 (The Pile)

Max Activating Token Index: 744

Text #18

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.9158. Min Act: -0.0000

Data Index: 1413250 (The Pile)

Max Activating Token Index: 899

Text #19

Max Range: 1.1592. Min Range: -1.1592

Max Act: 0.7815. Min Act: -0.0000

Data Index: 711343 (The Pile)

Max Activating Token Index: 16