Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1757 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.9607. Min Act: -0.0000

Data Index: 1073261 (The Pile)

Max Activating Token Index: 819

Text #1

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.7885. Min Act: -0.0000

Data Index: 1724190 (The Pile)

Max Activating Token Index: 65

Text #2

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.9543. Min Act: -0.0000

Data Index: 1691615 (The Pile)

Max Activating Token Index: 985

Text #3

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.7555. Min Act: -0.0000

Data Index: 793358 (The Pile)

Max Activating Token Index: 70

Text #4

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.5874. Min Act: -0.0000

Data Index: 496732 (The Pile)

Max Activating Token Index: 604

Text #5

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.4968. Min Act: -0.0000

Data Index: 602334 (The Pile)

Max Activating Token Index: 425

Text #6

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.5865. Min Act: -0.0000

Data Index: 3861 (The Pile)

Max Activating Token Index: 652

Text #7

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.3762. Min Act: -0.0000

Data Index: 1655704 (The Pile)

Max Activating Token Index: 600

Text #8

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.5209. Min Act: -0.0000

Data Index: 1224190 (The Pile)

Max Activating Token Index: 871

Text #9

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.0158. Min Act: -0.0000

Data Index: 507683 (The Pile)

Max Activating Token Index: 358

Text #10

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.3300. Min Act: -0.0000

Data Index: 1238497 (The Pile)

Max Activating Token Index: 332

Text #11

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.1892. Min Act: -0.0000

Data Index: 1579474 (The Pile)

Max Activating Token Index: 852

Text #12

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.3216. Min Act: -0.0000

Data Index: 180058 (The Pile)

Max Activating Token Index: 73

Text #13

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.5408. Min Act: -0.0000

Data Index: 843241 (The Pile)

Max Activating Token Index: 330

Text #14

Max Range: 3.9607. Min Range: -3.9607

Max Act: 2.9395. Min Act: -0.0000

Data Index: 736638 (The Pile)

Max Activating Token Index: 158

Text #15

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.0388. Min Act: -0.0000

Data Index: 38776 (The Pile)

Max Activating Token Index: 723

Text #16

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.1216. Min Act: -0.0000

Data Index: 1600116 (The Pile)

Max Activating Token Index: 257

Text #17

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.1156. Min Act: -0.0000

Data Index: 1535297 (The Pile)

Max Activating Token Index: 585

Text #18

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.0707. Min Act: -0.0000

Data Index: 1320256 (The Pile)

Max Activating Token Index: 34

Text #19

Max Range: 3.9607. Min Range: -3.9607

Max Act: 3.2188. Min Act: -0.0000

Data Index: 312996 (The Pile)

Max Activating Token Index: 700