Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3775 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.6756. Min Act: -0.0000

Data Index: 806673 (The Pile)

Max Activating Token Index: 248

Text #1

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.6190. Min Act: -0.0000

Data Index: 1112331 (The Pile)

Max Activating Token Index: 73

Text #2

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5855. Min Act: -0.0000

Data Index: 1994549 (The Pile)

Max Activating Token Index: 49

Text #3

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.6058. Min Act: -0.0000

Data Index: 133090 (The Pile)

Max Activating Token Index: 169

Text #4

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5826. Min Act: -0.0000

Data Index: 122099 (The Pile)

Max Activating Token Index: 972

Text #5

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5885. Min Act: -0.0000

Data Index: 327725 (The Pile)

Max Activating Token Index: 904

Text #6

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5763. Min Act: -0.0000

Data Index: 1065997 (The Pile)

Max Activating Token Index: 234

Text #7

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5786. Min Act: -0.0000

Data Index: 1364496 (The Pile)

Max Activating Token Index: 42

Text #8

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5372. Min Act: -0.0000

Data Index: 810323 (The Pile)

Max Activating Token Index: 573

Text #9

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5630. Min Act: -0.0000

Data Index: 1795730 (The Pile)

Max Activating Token Index: 609

Text #10

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5492. Min Act: -0.0000

Data Index: 1753095 (The Pile)

Max Activating Token Index: 827

Text #11

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5449. Min Act: -0.0000

Data Index: 736637 (The Pile)

Max Activating Token Index: 758

Text #12

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.4853. Min Act: -0.0000

Data Index: 1976400 (The Pile)

Max Activating Token Index: 970

Text #13

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.4845. Min Act: -0.0000

Data Index: 720991 (The Pile)

Max Activating Token Index: 695

Text #14

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5316. Min Act: -0.0000

Data Index: 1592213 (The Pile)

Max Activating Token Index: 16

Text #15

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.4695. Min Act: -0.0000

Data Index: 883010 (The Pile)

Max Activating Token Index: 1005

Text #16

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5655. Min Act: -0.0000

Data Index: 98552 (The Pile)

Max Activating Token Index: 866

Text #17

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5732. Min Act: -0.0000

Data Index: 1065339 (The Pile)

Max Activating Token Index: 47

Text #18

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.5916. Min Act: -0.0000

Data Index: 1193086 (The Pile)

Max Activating Token Index: 751

Text #19

Max Range: 0.6756. Min Range: -0.6756

Max Act: 0.4770. Min Act: -0.0000

Data Index: 1560329 (The Pile)

Max Activating Token Index: 140