Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2951 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.9262. Min Act: -0.0000

Data Index: 1248184 (The Pile)

Max Activating Token Index: 105

Text #1

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8619. Min Act: -0.0000

Data Index: 44610 (The Pile)

Max Activating Token Index: 100

Text #2

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8516. Min Act: -0.0000

Data Index: 1178977 (The Pile)

Max Activating Token Index: 882

Text #3

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8077. Min Act: -0.0000

Data Index: 366931 (The Pile)

Max Activating Token Index: 848

Text #4

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8136. Min Act: -0.0000

Data Index: 261461 (The Pile)

Max Activating Token Index: 329

Text #5

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8730. Min Act: -0.0000

Data Index: 445061 (The Pile)

Max Activating Token Index: 545

Text #6

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7555. Min Act: -0.0000

Data Index: 1600019 (The Pile)

Max Activating Token Index: 748

Text #7

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8417. Min Act: -0.0000

Data Index: 1969706 (The Pile)

Max Activating Token Index: 321

Text #8

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7924. Min Act: -0.0000

Data Index: 342926 (The Pile)

Max Activating Token Index: 148

Text #9

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.8188. Min Act: -0.0000

Data Index: 1759802 (The Pile)

Max Activating Token Index: 166

Text #10

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7679. Min Act: -0.0000

Data Index: 167043 (The Pile)

Max Activating Token Index: 656

Text #11

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7926. Min Act: -0.0000

Data Index: 1927595 (The Pile)

Max Activating Token Index: 880

Text #12

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7176. Min Act: -0.0000

Data Index: 1393529 (The Pile)

Max Activating Token Index: 705

Text #13

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7493. Min Act: -0.0000

Data Index: 1799624 (The Pile)

Max Activating Token Index: 1006

Text #14

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7712. Min Act: -0.0000

Data Index: 998561 (The Pile)

Max Activating Token Index: 324

Text #15

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7489. Min Act: -0.0000

Data Index: 1141699 (The Pile)

Max Activating Token Index: 998

Text #16

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7737. Min Act: -0.0000

Data Index: 1036447 (The Pile)

Max Activating Token Index: 589

Text #17

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7277. Min Act: -0.0000

Data Index: 1118501 (The Pile)

Max Activating Token Index: 800

Text #18

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.7959. Min Act: -0.0000

Data Index: 267128 (The Pile)

Max Activating Token Index: 802

Text #19

Max Range: 0.9262. Min Range: -0.9262

Max Act: 0.6753. Min Act: -0.0000

Data Index: 192099 (The Pile)

Max Activating Token Index: 971