Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3260 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.9297. Min Act: -0.0000

Data Index: 1738531 (The Pile)

Max Activating Token Index: 343

Text #1

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.8694. Min Act: -0.0000

Data Index: 699144 (The Pile)

Max Activating Token Index: 668

Text #2

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.8415. Min Act: -0.0000

Data Index: 883718 (The Pile)

Max Activating Token Index: 880

Text #3

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.8020. Min Act: -0.0000

Data Index: 295215 (The Pile)

Max Activating Token Index: 1022

Text #4

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.7694. Min Act: -0.0000

Data Index: 1993694 (The Pile)

Max Activating Token Index: 943

Text #5

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6922. Min Act: -0.0000

Data Index: 653579 (The Pile)

Max Activating Token Index: 927

Text #6

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6647. Min Act: -0.0000

Data Index: 1004422 (The Pile)

Max Activating Token Index: 59

Text #7

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.5966. Min Act: -0.0000

Data Index: 1519139 (The Pile)

Max Activating Token Index: 977

Text #8

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6659. Min Act: -0.0000

Data Index: 158792 (The Pile)

Max Activating Token Index: 394

Text #9

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6579. Min Act: -0.0000

Data Index: 1838118 (The Pile)

Max Activating Token Index: 616

Text #10

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6314. Min Act: -0.0000

Data Index: 971013 (The Pile)

Max Activating Token Index: 545

Text #11

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6175. Min Act: -0.0000

Data Index: 1643314 (The Pile)

Max Activating Token Index: 588

Text #12

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6199. Min Act: -0.0000

Data Index: 483631 (The Pile)

Max Activating Token Index: 519

Text #13

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6105. Min Act: -0.0000

Data Index: 1639020 (The Pile)

Max Activating Token Index: 459

Text #14

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6064. Min Act: -0.0000

Data Index: 1056888 (The Pile)

Max Activating Token Index: 252

Text #15

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.5637. Min Act: -0.0000

Data Index: 1458200 (The Pile)

Max Activating Token Index: 530

Text #16

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6024. Min Act: -0.0000

Data Index: 200151 (The Pile)

Max Activating Token Index: 374

Text #17

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6198. Min Act: -0.0000

Data Index: 1737159 (The Pile)

Max Activating Token Index: 161

Text #18

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.6179. Min Act: -0.0000

Data Index: 1479421 (The Pile)

Max Activating Token Index: 399

Text #19

Max Range: 0.9297. Min Range: -0.9297

Max Act: 0.5910. Min Act: -0.0000

Data Index: 1077394 (The Pile)

Max Activating Token Index: 468