Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2522 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 7.9213. Min Range: -7.9213

Max Act: 7.9213. Min Act: -0.0000

Data Index: 606044 (The Pile)

Max Activating Token Index: 755

Text #1

Max Range: 7.9213. Min Range: -7.9213

Max Act: 7.3698. Min Act: -0.0000

Data Index: 539619 (The Pile)

Max Activating Token Index: 1001

Text #2

Max Range: 7.9213. Min Range: -7.9213

Max Act: 7.1492. Min Act: -0.0000

Data Index: 621969 (The Pile)

Max Activating Token Index: 485

Text #3

Max Range: 7.9213. Min Range: -7.9213

Max Act: 7.0542. Min Act: -0.0000

Data Index: 1926419 (The Pile)

Max Activating Token Index: 954

Text #4

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.5359. Min Act: -0.0000

Data Index: 1841386 (The Pile)

Max Activating Token Index: 861

Text #5

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.8958. Min Act: -0.0000

Data Index: 715545 (The Pile)

Max Activating Token Index: 923

Text #6

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.8268. Min Act: -0.0000

Data Index: 1333742 (The Pile)

Max Activating Token Index: 774

Text #7

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.7546. Min Act: -0.0000

Data Index: 1560512 (The Pile)

Max Activating Token Index: 503

Text #8

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.6650. Min Act: -0.0000

Data Index: 1002710 (The Pile)

Max Activating Token Index: 803

Text #9

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.7018. Min Act: -0.0000

Data Index: 1625055 (The Pile)

Max Activating Token Index: 515

Text #10

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.6491. Min Act: -0.0000

Data Index: 233294 (The Pile)

Max Activating Token Index: 677

Text #11

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.6059. Min Act: -0.0000

Data Index: 1039742 (The Pile)

Max Activating Token Index: 407

Text #12

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.3996. Min Act: -0.0000

Data Index: 698063 (The Pile)

Max Activating Token Index: 1002

Text #13

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.1599. Min Act: -0.0000

Data Index: 1492920 (The Pile)

Max Activating Token Index: 863

Text #14

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.4485. Min Act: -0.0000

Data Index: 505675 (The Pile)

Max Activating Token Index: 600

Text #15

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.4644. Min Act: -0.0000

Data Index: 1258325 (The Pile)

Max Activating Token Index: 794

Text #16

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.3938. Min Act: -0.0000

Data Index: 1992167 (The Pile)

Max Activating Token Index: 343

Text #17

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.3955. Min Act: -0.0000

Data Index: 1251448 (The Pile)

Max Activating Token Index: 544

Text #18

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.2835. Min Act: -0.0000

Data Index: 1994710 (The Pile)

Max Activating Token Index: 408

Text #19

Max Range: 7.9213. Min Range: -7.9213

Max Act: 6.6107. Min Act: -0.0000

Data Index: 313376 (The Pile)

Max Activating Token Index: 922