Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 239 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3694. Min Act: -0.0000

Data Index: 174717 (The Pile)

Max Activating Token Index: 179

Text #1

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2947. Min Act: -0.0000

Data Index: 1134089 (The Pile)

Max Activating Token Index: 1010

Text #2

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3190. Min Act: -0.0000

Data Index: 948293 (The Pile)

Max Activating Token Index: 384

Text #3

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3359. Min Act: -0.0000

Data Index: 534714 (The Pile)

Max Activating Token Index: 808

Text #4

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2948. Min Act: -0.0000

Data Index: 1496548 (The Pile)

Max Activating Token Index: 869

Text #5

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3102. Min Act: -0.0000

Data Index: 1024513 (The Pile)

Max Activating Token Index: 625

Text #6

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3142. Min Act: -0.0000

Data Index: 362790 (The Pile)

Max Activating Token Index: 392

Text #7

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2691. Min Act: -0.0000

Data Index: 590613 (The Pile)

Max Activating Token Index: 101

Text #8

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2753. Min Act: -0.0000

Data Index: 1842402 (The Pile)

Max Activating Token Index: 221

Text #9

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3070. Min Act: -0.0000

Data Index: 1839795 (The Pile)

Max Activating Token Index: 229

Text #10

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2793. Min Act: -0.0000

Data Index: 668823 (The Pile)

Max Activating Token Index: 968

Text #11

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3059. Min Act: -0.0000

Data Index: 156467 (The Pile)

Max Activating Token Index: 112

Text #12

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2734. Min Act: -0.0000

Data Index: 458231 (The Pile)

Max Activating Token Index: 857

Text #13

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3003. Min Act: -0.0000

Data Index: 188719 (The Pile)

Max Activating Token Index: 626

Text #14

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3053. Min Act: -0.0000

Data Index: 582751 (The Pile)

Max Activating Token Index: 102

Text #15

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2749. Min Act: -0.0000

Data Index: 1547695 (The Pile)

Max Activating Token Index: 976

Text #16

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2760. Min Act: -0.0000

Data Index: 1439399 (The Pile)

Max Activating Token Index: 580

Text #17

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2648. Min Act: -0.0000

Data Index: 1720306 (The Pile)

Max Activating Token Index: 389

Text #18

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.3026. Min Act: -0.0000

Data Index: 1969553 (The Pile)

Max Activating Token Index: 379

Text #19

Max Range: 0.3694. Min Range: -0.3694

Max Act: 0.2586. Min Act: -0.0000

Data Index: 121978 (The Pile)

Max Activating Token Index: 599