Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 941 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.4075. Min Range: -5.4075

Max Act: 5.4075. Min Act: -0.0000

Data Index: 1482167 (The Pile)

Max Activating Token Index: 224

Text #1

Max Range: 5.4075. Min Range: -5.4075

Max Act: 4.5576. Min Act: -0.0000

Data Index: 439626 (The Pile)

Max Activating Token Index: 601

Text #2

Max Range: 5.4075. Min Range: -5.4075

Max Act: 4.3752. Min Act: -0.0000

Data Index: 1815608 (The Pile)

Max Activating Token Index: 211

Text #3

Max Range: 5.4075. Min Range: -5.4075

Max Act: 4.1891. Min Act: -0.0000

Data Index: 1088534 (The Pile)

Max Activating Token Index: 606

Text #4

Max Range: 5.4075. Min Range: -5.4075

Max Act: 4.1352. Min Act: -0.0000

Data Index: 1696822 (The Pile)

Max Activating Token Index: 543

Text #5

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.9807. Min Act: -0.0000

Data Index: 1402081 (The Pile)

Max Activating Token Index: 333

Text #6

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.8683. Min Act: -0.0000

Data Index: 434237 (The Pile)

Max Activating Token Index: 275

Text #7

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.5911. Min Act: -0.0000

Data Index: 1577348 (The Pile)

Max Activating Token Index: 636

Text #8

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.3601. Min Act: -0.0000

Data Index: 473214 (The Pile)

Max Activating Token Index: 893

Text #9

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.3384. Min Act: -0.0000

Data Index: 76228 (The Pile)

Max Activating Token Index: 182

Text #10

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.5122. Min Act: -0.0000

Data Index: 1950530 (The Pile)

Max Activating Token Index: 612

Text #11

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.1548. Min Act: -0.0000

Data Index: 1646068 (The Pile)

Max Activating Token Index: 205

Text #12

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.0097. Min Act: -0.0000

Data Index: 1087063 (The Pile)

Max Activating Token Index: 238

Text #13

Max Range: 5.4075. Min Range: -5.4075

Max Act: 2.8695. Min Act: -0.0000

Data Index: 1692484 (The Pile)

Max Activating Token Index: 663

Text #14

Max Range: 5.4075. Min Range: -5.4075

Max Act: 2.5599. Min Act: -0.0000

Data Index: 1868254 (The Pile)

Max Activating Token Index: 965

Text #15

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.1885. Min Act: -0.0000

Data Index: 741731 (The Pile)

Max Activating Token Index: 630

Text #16

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.0432. Min Act: -0.0000

Data Index: 276455 (The Pile)

Max Activating Token Index: 837

Text #17

Max Range: 5.4075. Min Range: -5.4075

Max Act: 3.0985. Min Act: -0.0000

Data Index: 990245 (The Pile)

Max Activating Token Index: 514

Text #18

Max Range: 5.4075. Min Range: -5.4075

Max Act: 2.8803. Min Act: -0.0000

Data Index: 1683035 (The Pile)

Max Activating Token Index: 593

Text #19

Max Range: 5.4075. Min Range: -5.4075

Max Act: 2.8715. Min Act: -0.0000

Data Index: 1924535 (The Pile)

Max Activating Token Index: 466