Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4333 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2940. Min Act: -0.0000

Data Index: 9726 (The Pile)

Max Activating Token Index: 91

Text #1

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2811. Min Act: -0.0000

Data Index: 602095 (The Pile)

Max Activating Token Index: 78

Text #2

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2760. Min Act: -0.0000

Data Index: 1973277 (The Pile)

Max Activating Token Index: 676

Text #3

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2621. Min Act: -0.0000

Data Index: 1374837 (The Pile)

Max Activating Token Index: 729

Text #4

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2620. Min Act: -0.0000

Data Index: 1422514 (The Pile)

Max Activating Token Index: 409

Text #5

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2503. Min Act: -0.0000

Data Index: 214382 (The Pile)

Max Activating Token Index: 228

Text #6

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2635. Min Act: -0.0000

Data Index: 487623 (The Pile)

Max Activating Token Index: 403

Text #7

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2594. Min Act: -0.0000

Data Index: 748633 (The Pile)

Max Activating Token Index: 271

Text #8

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2356. Min Act: -0.0000

Data Index: 1614637 (The Pile)

Max Activating Token Index: 133

Text #9

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2252. Min Act: -0.0000

Data Index: 1559868 (The Pile)

Max Activating Token Index: 674

Text #10

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2420. Min Act: -0.0000

Data Index: 83570 (The Pile)

Max Activating Token Index: 937

Text #11

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2444. Min Act: -0.0000

Data Index: 660420 (The Pile)

Max Activating Token Index: 469

Text #12

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2399. Min Act: -0.0000

Data Index: 844147 (The Pile)

Max Activating Token Index: 163

Text #13

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2434. Min Act: -0.0000

Data Index: 29160 (The Pile)

Max Activating Token Index: 187

Text #14

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2259. Min Act: -0.0000

Data Index: 72520 (The Pile)

Max Activating Token Index: 561

Text #15

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2398. Min Act: -0.0000

Data Index: 792713 (The Pile)

Max Activating Token Index: 465

Text #16

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2474. Min Act: -0.0000

Data Index: 1833018 (The Pile)

Max Activating Token Index: 410

Text #17

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2458. Min Act: -0.0000

Data Index: 1466730 (The Pile)

Max Activating Token Index: 186

Text #18

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2339. Min Act: -0.0000

Data Index: 1506170 (The Pile)

Max Activating Token Index: 380

Text #19

Max Range: 0.2940. Min Range: -0.2940

Max Act: 0.2356. Min Act: -0.0000

Data Index: 365075 (The Pile)

Max Activating Token Index: 306