Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4194 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.2854. Min Act: -0.0000

Data Index: 575464 (The Pile)

Max Activating Token Index: 527

Text #1

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.2311. Min Act: -0.0000

Data Index: 1612837 (The Pile)

Max Activating Token Index: 968

Text #2

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.2220. Min Act: -0.0000

Data Index: 1349923 (The Pile)

Max Activating Token Index: 502

Text #3

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1978. Min Act: -0.0000

Data Index: 282300 (The Pile)

Max Activating Token Index: 716

Text #4

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.2981. Min Act: -0.0000

Data Index: 1779087 (The Pile)

Max Activating Token Index: 86

Text #5

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1580. Min Act: -0.0000

Data Index: 1983047 (The Pile)

Max Activating Token Index: 710

Text #6

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1466. Min Act: -0.0000

Data Index: 1074849 (The Pile)

Max Activating Token Index: 526

Text #7

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0838. Min Act: -0.0000

Data Index: 145483 (The Pile)

Max Activating Token Index: 736

Text #8

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1289. Min Act: -0.0000

Data Index: 1638480 (The Pile)

Max Activating Token Index: 334

Text #9

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0609. Min Act: -0.0000

Data Index: 1591276 (The Pile)

Max Activating Token Index: 295

Text #10

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1509. Min Act: -0.0000

Data Index: 1015795 (The Pile)

Max Activating Token Index: 642

Text #11

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0545. Min Act: -0.0000

Data Index: 415588 (The Pile)

Max Activating Token Index: 438

Text #12

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0193. Min Act: -0.0000

Data Index: 1866951 (The Pile)

Max Activating Token Index: 147

Text #13

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.1114. Min Act: -0.0000

Data Index: 1339844 (The Pile)

Max Activating Token Index: 926

Text #14

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0552. Min Act: -0.0000

Data Index: 143247 (The Pile)

Max Activating Token Index: 183

Text #15

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0035. Min Act: -0.0000

Data Index: 956484 (The Pile)

Max Activating Token Index: 716

Text #16

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0469. Min Act: -0.0000

Data Index: 301383 (The Pile)

Max Activating Token Index: 399

Text #17

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0459. Min Act: -0.0000

Data Index: 392038 (The Pile)

Max Activating Token Index: 660

Text #18

Max Range: 1.2981. Min Range: -1.2981

Max Act: 1.0321. Min Act: -0.0000

Data Index: 1158479 (The Pile)

Max Activating Token Index: 998

Text #19

Max Range: 1.2981. Min Range: -1.2981

Max Act: 0.9395. Min Act: -0.0000

Data Index: 711634 (The Pile)

Max Activating Token Index: 306