Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2102 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.5646. Min Act: -0.0000

Data Index: 1488576 (The Pile)

Max Activating Token Index: 861

Text #1

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.5581. Min Act: -0.0000

Data Index: 1329754 (The Pile)

Max Activating Token Index: 480

Text #2

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.3642. Min Act: -0.0000

Data Index: 1982799 (The Pile)

Max Activating Token Index: 417

Text #3

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.2081. Min Act: -0.0000

Data Index: 385001 (The Pile)

Max Activating Token Index: 775

Text #4

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.2737. Min Act: -0.0000

Data Index: 1296005 (The Pile)

Max Activating Token Index: 1021

Text #5

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.1253. Min Act: -0.0000

Data Index: 1045612 (The Pile)

Max Activating Token Index: 574

Text #6

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9414. Min Act: -0.0000

Data Index: 90115 (The Pile)

Max Activating Token Index: 198

Text #7

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0919. Min Act: -0.0000

Data Index: 1733031 (The Pile)

Max Activating Token Index: 276

Text #8

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0369. Min Act: -0.0000

Data Index: 460282 (The Pile)

Max Activating Token Index: 111

Text #9

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0849. Min Act: -0.0000

Data Index: 1237519 (The Pile)

Max Activating Token Index: 354

Text #10

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0048. Min Act: -0.0000

Data Index: 552088 (The Pile)

Max Activating Token Index: 786

Text #11

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9679. Min Act: -0.0000

Data Index: 937338 (The Pile)

Max Activating Token Index: 589

Text #12

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0208. Min Act: -0.0000

Data Index: 1004327 (The Pile)

Max Activating Token Index: 197

Text #13

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0060. Min Act: -0.0000

Data Index: 738799 (The Pile)

Max Activating Token Index: 940

Text #14

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9953. Min Act: -0.0000

Data Index: 1985700 (The Pile)

Max Activating Token Index: 388

Text #15

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9867. Min Act: -0.0000

Data Index: 1972166 (The Pile)

Max Activating Token Index: 789

Text #16

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9600. Min Act: -0.0000

Data Index: 886375 (The Pile)

Max Activating Token Index: 708

Text #17

Max Range: 1.5646. Min Range: -1.5646

Max Act: 1.0011. Min Act: -0.0000

Data Index: 525329 (The Pile)

Max Activating Token Index: 862

Text #18

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9011. Min Act: -0.0000

Data Index: 1412765 (The Pile)

Max Activating Token Index: 63

Text #19

Max Range: 1.5646. Min Range: -1.5646

Max Act: 0.9776. Min Act: -0.0000

Data Index: 715946 (The Pile)

Max Activating Token Index: 837