Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2492 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6403. Min Act: -0.0000

Data Index: 1990658 (The Pile)

Max Activating Token Index: 323

Text #1

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6055. Min Act: -0.0000

Data Index: 265696 (The Pile)

Max Activating Token Index: 912

Text #2

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6225. Min Act: -0.0000

Data Index: 710526 (The Pile)

Max Activating Token Index: 534

Text #3

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5703. Min Act: -0.0000

Data Index: 870733 (The Pile)

Max Activating Token Index: 210

Text #4

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6007. Min Act: -0.0000

Data Index: 1373908 (The Pile)

Max Activating Token Index: 220

Text #5

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6097. Min Act: -0.0000

Data Index: 635851 (The Pile)

Max Activating Token Index: 275

Text #6

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5302. Min Act: -0.0000

Data Index: 290012 (The Pile)

Max Activating Token Index: 381

Text #7

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.6162. Min Act: -0.0000

Data Index: 676906 (The Pile)

Max Activating Token Index: 891

Text #8

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5422. Min Act: -0.0000

Data Index: 1694372 (The Pile)

Max Activating Token Index: 646

Text #9

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5563. Min Act: -0.0000

Data Index: 873381 (The Pile)

Max Activating Token Index: 257

Text #10

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5520. Min Act: -0.0000

Data Index: 373881 (The Pile)

Max Activating Token Index: 279

Text #11

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5369. Min Act: -0.0000

Data Index: 732763 (The Pile)

Max Activating Token Index: 415

Text #12

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5479. Min Act: -0.0000

Data Index: 872211 (The Pile)

Max Activating Token Index: 957

Text #13

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.4856. Min Act: -0.0000

Data Index: 1171088 (The Pile)

Max Activating Token Index: 620

Text #14

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5346. Min Act: -0.0000

Data Index: 779260 (The Pile)

Max Activating Token Index: 115

Text #15

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5033. Min Act: -0.0000

Data Index: 1861675 (The Pile)

Max Activating Token Index: 574

Text #16

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5001. Min Act: -0.0000

Data Index: 1893024 (The Pile)

Max Activating Token Index: 165

Text #17

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5320. Min Act: -0.0000

Data Index: 629111 (The Pile)

Max Activating Token Index: 168

Text #18

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.5627. Min Act: -0.0000

Data Index: 965321 (The Pile)

Max Activating Token Index: 770

Text #19

Max Range: 0.6403. Min Range: -0.6403

Max Act: 0.4886. Min Act: -0.0000

Data Index: 1105102 (The Pile)

Max Activating Token Index: 643