Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4289 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.7205. Min Act: -0.0000

Data Index: 1483561 (The Pile)

Max Activating Token Index: 632

Text #1

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.5907. Min Act: -0.0000

Data Index: 1327542 (The Pile)

Max Activating Token Index: 628

Text #2

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.5059. Min Act: -0.0000

Data Index: 679657 (The Pile)

Max Activating Token Index: 466

Text #3

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.2635. Min Act: -0.0000

Data Index: 414300 (The Pile)

Max Activating Token Index: 99

Text #4

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.2227. Min Act: -0.0000

Data Index: 955030 (The Pile)

Max Activating Token Index: 258

Text #5

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.1485. Min Act: -0.0000

Data Index: 1467235 (The Pile)

Max Activating Token Index: 290

Text #6

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.1027. Min Act: -0.0000

Data Index: 1603060 (The Pile)

Max Activating Token Index: 588

Text #7

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.9319. Min Act: -0.0000

Data Index: 1689311 (The Pile)

Max Activating Token Index: 475

Text #8

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.9396. Min Act: -0.0000

Data Index: 1179949 (The Pile)

Max Activating Token Index: 799

Text #9

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.2682. Min Act: -0.0000

Data Index: 936951 (The Pile)

Max Activating Token Index: 525

Text #10

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.9071. Min Act: -0.0000

Data Index: 597251 (The Pile)

Max Activating Token Index: 211

Text #11

Max Range: 3.7205. Min Range: -3.7205

Max Act: 3.0030. Min Act: -0.0000

Data Index: 914118 (The Pile)

Max Activating Token Index: 988

Text #12

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.8806. Min Act: -0.0000

Data Index: 1420463 (The Pile)

Max Activating Token Index: 540

Text #13

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.6993. Min Act: -0.0000

Data Index: 273722 (The Pile)

Max Activating Token Index: 546

Text #14

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.8219. Min Act: -0.0000

Data Index: 123386 (The Pile)

Max Activating Token Index: 640

Text #15

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.8075. Min Act: -0.0000

Data Index: 1139695 (The Pile)

Max Activating Token Index: 754

Text #16

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.8099. Min Act: -0.0000

Data Index: 1912393 (The Pile)

Max Activating Token Index: 47

Text #17

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.6445. Min Act: -0.0000

Data Index: 1761084 (The Pile)

Max Activating Token Index: 531

Text #18

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.5855. Min Act: -0.0000

Data Index: 1668756 (The Pile)

Max Activating Token Index: 618

Text #19

Max Range: 3.7205. Min Range: -3.7205

Max Act: 2.7229. Min Act: -0.0000

Data Index: 1313250 (The Pile)

Max Activating Token Index: 589