Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 292 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0195. Min Range: -1.0195

Max Act: 1.0011. Min Act: -0.0000

Data Index: 611526 (The Pile)

Max Activating Token Index: 676

Text #1

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9911. Min Act: -0.0000

Data Index: 1181842 (The Pile)

Max Activating Token Index: 655

Text #2

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9288. Min Act: -0.0000

Data Index: 386010 (The Pile)

Max Activating Token Index: 114

Text #3

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9633. Min Act: -0.0000

Data Index: 1900699 (The Pile)

Max Activating Token Index: 730

Text #4

Max Range: 1.0195. Min Range: -1.0195

Max Act: 1.0195. Min Act: -0.0000

Data Index: 131326 (The Pile)

Max Activating Token Index: 254

Text #5

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9255. Min Act: -0.0000

Data Index: 1073527 (The Pile)

Max Activating Token Index: 351

Text #6

Max Range: 1.0195. Min Range: -1.0195

Max Act: 1.0012. Min Act: -0.0000

Data Index: 472432 (The Pile)

Max Activating Token Index: 646

Text #7

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9602. Min Act: -0.0000

Data Index: 199378 (The Pile)

Max Activating Token Index: 307

Text #8

Max Range: 1.0195. Min Range: -1.0195

Max Act: 1.0089. Min Act: -0.0000

Data Index: 777558 (The Pile)

Max Activating Token Index: 276

Text #9

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.8996. Min Act: -0.0000

Data Index: 56102 (The Pile)

Max Activating Token Index: 923

Text #10

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9160. Min Act: -0.0000

Data Index: 601158 (The Pile)

Max Activating Token Index: 173

Text #11

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.8981. Min Act: -0.0000

Data Index: 206874 (The Pile)

Max Activating Token Index: 205

Text #12

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9296. Min Act: -0.0000

Data Index: 607261 (The Pile)

Max Activating Token Index: 587

Text #13

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9544. Min Act: -0.0000

Data Index: 1063076 (The Pile)

Max Activating Token Index: 639

Text #14

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9524. Min Act: -0.0000

Data Index: 1644499 (The Pile)

Max Activating Token Index: 585

Text #15

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9320. Min Act: -0.0000

Data Index: 173980 (The Pile)

Max Activating Token Index: 758

Text #16

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9016. Min Act: -0.0000

Data Index: 1360789 (The Pile)

Max Activating Token Index: 760

Text #17

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.9273. Min Act: -0.0000

Data Index: 1609933 (The Pile)

Max Activating Token Index: 328

Text #18

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.8537. Min Act: -0.0000

Data Index: 1413810 (The Pile)

Max Activating Token Index: 870

Text #19

Max Range: 1.0195. Min Range: -1.0195

Max Act: 0.8867. Min Act: -0.0000

Data Index: 1831857 (The Pile)

Max Activating Token Index: 874