Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 79 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1105. Min Range: -1.1105

Max Act: 1.1105. Min Act: -0.0000

Data Index: 184612 (The Pile)

Max Activating Token Index: 291

Text #1

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.7853. Min Act: -0.0000

Data Index: 276370 (The Pile)

Max Activating Token Index: 331

Text #2

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.7038. Min Act: -0.0000

Data Index: 1849868 (The Pile)

Max Activating Token Index: 630

Text #3

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6920. Min Act: -0.0000

Data Index: 398611 (The Pile)

Max Activating Token Index: 545

Text #4

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.7344. Min Act: -0.0000

Data Index: 869596 (The Pile)

Max Activating Token Index: 190

Text #5

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.8207. Min Act: -0.0000

Data Index: 607626 (The Pile)

Max Activating Token Index: 1020

Text #6

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6478. Min Act: -0.0000

Data Index: 580922 (The Pile)

Max Activating Token Index: 797

Text #7

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6347. Min Act: -0.0000

Data Index: 1279221 (The Pile)

Max Activating Token Index: 465

Text #8

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.7042. Min Act: -0.0000

Data Index: 1668231 (The Pile)

Max Activating Token Index: 883

Text #9

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6206. Min Act: -0.0000

Data Index: 1971066 (The Pile)

Max Activating Token Index: 519

Text #10

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6273. Min Act: -0.0000

Data Index: 1266798 (The Pile)

Max Activating Token Index: 144

Text #11

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6906. Min Act: -0.0000

Data Index: 134330 (The Pile)

Max Activating Token Index: 418

Text #12

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6357. Min Act: -0.0000

Data Index: 24029 (The Pile)

Max Activating Token Index: 254

Text #13

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6515. Min Act: -0.0000

Data Index: 1480331 (The Pile)

Max Activating Token Index: 595

Text #14

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6012. Min Act: -0.0000

Data Index: 1944446 (The Pile)

Max Activating Token Index: 70

Text #15

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6394. Min Act: -0.0000

Data Index: 8243 (The Pile)

Max Activating Token Index: 598

Text #16

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6349. Min Act: -0.0000

Data Index: 287642 (The Pile)

Max Activating Token Index: 376

Text #17

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6555. Min Act: -0.0000

Data Index: 732017 (The Pile)

Max Activating Token Index: 113

Text #18

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.6049. Min Act: -0.0000

Data Index: 197997 (The Pile)

Max Activating Token Index: 236

Text #19

Max Range: 1.1105. Min Range: -1.1105

Max Act: 0.5275. Min Act: -0.0000

Data Index: 777280 (The Pile)

Max Activating Token Index: 158