Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4747 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.3719. Min Act: -0.0000

Data Index: 730652 (The Pile)

Max Activating Token Index: 404

Text #1

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2947. Min Act: -0.0000

Data Index: 832930 (The Pile)

Max Activating Token Index: 919

Text #2

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2911. Min Act: -0.0000

Data Index: 774448 (The Pile)

Max Activating Token Index: 627

Text #3

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2816. Min Act: -0.0000

Data Index: 1026878 (The Pile)

Max Activating Token Index: 101

Text #4

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.3025. Min Act: -0.0000

Data Index: 1635662 (The Pile)

Max Activating Token Index: 139

Text #5

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2654. Min Act: -0.0000

Data Index: 1321578 (The Pile)

Max Activating Token Index: 1012

Text #6

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2605. Min Act: -0.0000

Data Index: 1867975 (The Pile)

Max Activating Token Index: 865

Text #7

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2548. Min Act: -0.0000

Data Index: 776094 (The Pile)

Max Activating Token Index: 242

Text #8

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2515. Min Act: -0.0000

Data Index: 470536 (The Pile)

Max Activating Token Index: 1001

Text #9

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2415. Min Act: -0.0000

Data Index: 1942078 (The Pile)

Max Activating Token Index: 535

Text #10

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2525. Min Act: -0.0000

Data Index: 1830355 (The Pile)

Max Activating Token Index: 947

Text #11

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2440. Min Act: -0.0000

Data Index: 194695 (The Pile)

Max Activating Token Index: 235

Text #12

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2604. Min Act: -0.0000

Data Index: 96697 (The Pile)

Max Activating Token Index: 676

Text #13

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2344. Min Act: -0.0000

Data Index: 266805 (The Pile)

Max Activating Token Index: 726

Text #14

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2427. Min Act: -0.0000

Data Index: 1282221 (The Pile)

Max Activating Token Index: 557

Text #15

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2351. Min Act: -0.0000

Data Index: 1672073 (The Pile)

Max Activating Token Index: 693

Text #16

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2454. Min Act: -0.0000

Data Index: 162503 (The Pile)

Max Activating Token Index: 882

Text #17

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2430. Min Act: -0.0000

Data Index: 1993461 (The Pile)

Max Activating Token Index: 926

Text #18

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2445. Min Act: -0.0000

Data Index: 88646 (The Pile)

Max Activating Token Index: 620

Text #19

Max Range: 0.3719. Min Range: -0.3719

Max Act: 0.2396. Min Act: -0.0000

Data Index: 1577109 (The Pile)

Max Activating Token Index: 790