Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2757 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.8282. Min Range: -4.8282

Max Act: 4.8282. Min Act: -0.0000

Data Index: 1970542 (The Pile)

Max Activating Token Index: 615

Text #1

Max Range: 4.8282. Min Range: -4.8282

Max Act: 4.6638. Min Act: -0.0000

Data Index: 1520254 (The Pile)

Max Activating Token Index: 389

Text #2

Max Range: 4.8282. Min Range: -4.8282

Max Act: 4.0884. Min Act: -0.0000

Data Index: 684262 (The Pile)

Max Activating Token Index: 725

Text #3

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.6687. Min Act: -0.0000

Data Index: 832488 (The Pile)

Max Activating Token Index: 866

Text #4

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.6435. Min Act: -0.0000

Data Index: 1015654 (The Pile)

Max Activating Token Index: 812

Text #5

Max Range: 4.8282. Min Range: -4.8282

Max Act: 4.1074. Min Act: -0.0000

Data Index: 614941 (The Pile)

Max Activating Token Index: 118

Text #6

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.6681. Min Act: -0.0000

Data Index: 1749999 (The Pile)

Max Activating Token Index: 778

Text #7

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.4684. Min Act: -0.0000

Data Index: 93724 (The Pile)

Max Activating Token Index: 619

Text #8

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.1170. Min Act: -0.0000

Data Index: 1686491 (The Pile)

Max Activating Token Index: 1015

Text #9

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.4203. Min Act: -0.0000

Data Index: 1370200 (The Pile)

Max Activating Token Index: 797

Text #10

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.2794. Min Act: -0.0000

Data Index: 1853322 (The Pile)

Max Activating Token Index: 515

Text #11

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.7281. Min Act: -0.0000

Data Index: 1020136 (The Pile)

Max Activating Token Index: 149

Text #12

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.3856. Min Act: -0.0000

Data Index: 1458291 (The Pile)

Max Activating Token Index: 87

Text #13

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.1447. Min Act: -0.0000

Data Index: 648876 (The Pile)

Max Activating Token Index: 648

Text #14

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.8561. Min Act: -0.0000

Data Index: 1381272 (The Pile)

Max Activating Token Index: 362

Text #15

Max Range: 4.8282. Min Range: -4.8282

Max Act: 3.0476. Min Act: -0.0000

Data Index: 1977145 (The Pile)

Max Activating Token Index: 601

Text #16

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.9596. Min Act: -0.0000

Data Index: 1675111 (The Pile)

Max Activating Token Index: 52

Text #17

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.4767. Min Act: -0.0000

Data Index: 1879367 (The Pile)

Max Activating Token Index: 322

Text #18

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.8457. Min Act: -0.0000

Data Index: 343170 (The Pile)

Max Activating Token Index: 59

Text #19

Max Range: 4.8282. Min Range: -4.8282

Max Act: 2.9071. Min Act: -0.0000

Data Index: 879311 (The Pile)

Max Activating Token Index: 931