Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 169 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.9516. Min Act: -0.0000

Data Index: 778449 (The Pile)

Max Activating Token Index: 888

Text #1

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.8503. Min Act: -0.0000

Data Index: 1117509 (The Pile)

Max Activating Token Index: 453

Text #2

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.9098. Min Act: -0.0000

Data Index: 771454 (The Pile)

Max Activating Token Index: 557

Text #3

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7247. Min Act: -0.0000

Data Index: 1739958 (The Pile)

Max Activating Token Index: 706

Text #4

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7577. Min Act: -0.0000

Data Index: 1096172 (The Pile)

Max Activating Token Index: 314

Text #5

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7601. Min Act: -0.0000

Data Index: 914200 (The Pile)

Max Activating Token Index: 577

Text #6

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7421. Min Act: -0.0000

Data Index: 513894 (The Pile)

Max Activating Token Index: 983

Text #7

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7151. Min Act: -0.0000

Data Index: 269010 (The Pile)

Max Activating Token Index: 956

Text #8

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7261. Min Act: -0.0000

Data Index: 32083 (The Pile)

Max Activating Token Index: 974

Text #9

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6982. Min Act: -0.0000

Data Index: 1425384 (The Pile)

Max Activating Token Index: 149

Text #10

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6521. Min Act: -0.0000

Data Index: 1027849 (The Pile)

Max Activating Token Index: 834

Text #11

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6678. Min Act: -0.0000

Data Index: 297955 (The Pile)

Max Activating Token Index: 677

Text #12

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6548. Min Act: -0.0000

Data Index: 1260021 (The Pile)

Max Activating Token Index: 896

Text #13

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.7068. Min Act: -0.0000

Data Index: 1305898 (The Pile)

Max Activating Token Index: 849

Text #14

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6517. Min Act: -0.0000

Data Index: 1709489 (The Pile)

Max Activating Token Index: 951

Text #15

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6434. Min Act: -0.0000

Data Index: 1414787 (The Pile)

Max Activating Token Index: 327

Text #16

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6875. Min Act: -0.0000

Data Index: 1657197 (The Pile)

Max Activating Token Index: 726

Text #17

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6372. Min Act: -0.0000

Data Index: 1124141 (The Pile)

Max Activating Token Index: 268

Text #18

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6545. Min Act: -0.0000

Data Index: 678113 (The Pile)

Max Activating Token Index: 867

Text #19

Max Range: 0.9516. Min Range: -0.9516

Max Act: 0.6565. Min Act: -0.0000

Data Index: 851357 (The Pile)

Max Activating Token Index: 893