Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4499 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.2311. Min Act: -0.0000

Data Index: 1793578 (The Pile)

Max Activating Token Index: 556

Text #1

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.2212. Min Act: -0.0000

Data Index: 1963285 (The Pile)

Max Activating Token Index: 179

Text #2

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.2448. Min Act: -0.0000

Data Index: 1151875 (The Pile)

Max Activating Token Index: 449

Text #3

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1700. Min Act: -0.0000

Data Index: 1850552 (The Pile)

Max Activating Token Index: 871

Text #4

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0812. Min Act: -0.0000

Data Index: 1730905 (The Pile)

Max Activating Token Index: 157

Text #5

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1681. Min Act: -0.0000

Data Index: 1851045 (The Pile)

Max Activating Token Index: 217

Text #6

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1351. Min Act: -0.0000

Data Index: 75233 (The Pile)

Max Activating Token Index: 107

Text #7

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0784. Min Act: -0.0000

Data Index: 1883799 (The Pile)

Max Activating Token Index: 545

Text #8

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1376. Min Act: -0.0000

Data Index: 1417705 (The Pile)

Max Activating Token Index: 498

Text #9

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1077. Min Act: -0.0000

Data Index: 1889353 (The Pile)

Max Activating Token Index: 234

Text #10

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1057. Min Act: -0.0000

Data Index: 1941127 (The Pile)

Max Activating Token Index: 430

Text #11

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.2345. Min Act: -0.0000

Data Index: 1111080 (The Pile)

Max Activating Token Index: 172

Text #12

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0596. Min Act: -0.0000

Data Index: 127019 (The Pile)

Max Activating Token Index: 1020

Text #13

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.2100. Min Act: -0.0000

Data Index: 128514 (The Pile)

Max Activating Token Index: 152

Text #14

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1257. Min Act: -0.0000

Data Index: 590453 (The Pile)

Max Activating Token Index: 540

Text #15

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0322. Min Act: -0.0000

Data Index: 781056 (The Pile)

Max Activating Token Index: 136

Text #16

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1027. Min Act: -0.0000

Data Index: 1028584 (The Pile)

Max Activating Token Index: 428

Text #17

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.1185. Min Act: -0.0000

Data Index: 236554 (The Pile)

Max Activating Token Index: 502

Text #18

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0930. Min Act: -0.0000

Data Index: 1825313 (The Pile)

Max Activating Token Index: 526

Text #19

Max Range: 1.2448. Min Range: -1.2448

Max Act: 1.0806. Min Act: -0.0000

Data Index: 1384994 (The Pile)

Max Activating Token Index: 311