Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3243 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0611. Min Range: -1.0611

Max Act: 1.0611. Min Act: -0.0000

Data Index: 1455920 (The Pile)

Max Activating Token Index: 887

Text #1

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.9085. Min Act: -0.0000

Data Index: 1846704 (The Pile)

Max Activating Token Index: 142

Text #2

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.8046. Min Act: -0.0000

Data Index: 1807022 (The Pile)

Max Activating Token Index: 43

Text #3

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.8958. Min Act: -0.0000

Data Index: 1758605 (The Pile)

Max Activating Token Index: 488

Text #4

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.7471. Min Act: 0.0000

Data Index: 1244032 (The Pile)

Max Activating Token Index: 48

Text #5

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.8445. Min Act: -0.0000

Data Index: 943016 (The Pile)

Max Activating Token Index: 122

Text #6

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.7839. Min Act: -0.0000

Data Index: 59596 (The Pile)

Max Activating Token Index: 21

Text #7

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.7165. Min Act: -0.0000

Data Index: 229376 (The Pile)

Max Activating Token Index: 641

Text #8

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6930. Min Act: -0.0000

Data Index: 1483416 (The Pile)

Max Activating Token Index: 423

Text #9

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6927. Min Act: -0.0000

Data Index: 405522 (The Pile)

Max Activating Token Index: 195

Text #10

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.7270. Min Act: -0.0000

Data Index: 235388 (The Pile)

Max Activating Token Index: 155

Text #11

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6035. Min Act: -0.0000

Data Index: 181968 (The Pile)

Max Activating Token Index: 787

Text #12

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6747. Min Act: -0.0000

Data Index: 1491139 (The Pile)

Max Activating Token Index: 654

Text #13

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6811. Min Act: -0.0000

Data Index: 1342580 (The Pile)

Max Activating Token Index: 669

Text #14

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6826. Min Act: -0.0000

Data Index: 1121000 (The Pile)

Max Activating Token Index: 633

Text #15

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6394. Min Act: -0.0000

Data Index: 609599 (The Pile)

Max Activating Token Index: 396

Text #16

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6222. Min Act: -0.0000

Data Index: 1324417 (The Pile)

Max Activating Token Index: 457

Text #17

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.5654. Min Act: -0.0000

Data Index: 425409 (The Pile)

Max Activating Token Index: 921

Text #18

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.6383. Min Act: -0.0000

Data Index: 1889180 (The Pile)

Max Activating Token Index: 206

Text #19

Max Range: 1.0611. Min Range: -1.0611

Max Act: 0.5872. Min Act: -0.0000

Data Index: 1215492 (The Pile)

Max Activating Token Index: 147