Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 726 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.8391. Min Act: -0.0000

Data Index: 715564 (The Pile)

Max Activating Token Index: 986

Text #1

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.5098. Min Act: -0.0000

Data Index: 1517912 (The Pile)

Max Activating Token Index: 193

Text #2

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.5409. Min Act: -0.0000

Data Index: 1208454 (The Pile)

Max Activating Token Index: 853

Text #3

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.2824. Min Act: -0.0000

Data Index: 1785998 (The Pile)

Max Activating Token Index: 108

Text #4

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.5313. Min Act: -0.0000

Data Index: 1335544 (The Pile)

Max Activating Token Index: 184

Text #5

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.2578. Min Act: -0.0000

Data Index: 153573 (The Pile)

Max Activating Token Index: 687

Text #6

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.3089. Min Act: -0.0000

Data Index: 189091 (The Pile)

Max Activating Token Index: 434

Text #7

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.3151. Min Act: -0.0000

Data Index: 1941361 (The Pile)

Max Activating Token Index: 192

Text #8

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1606. Min Act: 0.0000

Data Index: 745795 (The Pile)

Max Activating Token Index: 199

Text #9

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1734. Min Act: -0.0000

Data Index: 1092128 (The Pile)

Max Activating Token Index: 536

Text #10

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1852. Min Act: -0.0000

Data Index: 1345154 (The Pile)

Max Activating Token Index: 335

Text #11

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1752. Min Act: -0.0000

Data Index: 1026988 (The Pile)

Max Activating Token Index: 376

Text #12

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1596. Min Act: -0.0000

Data Index: 1515206 (The Pile)

Max Activating Token Index: 181

Text #13

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1046. Min Act: 0.0000

Data Index: 444421 (The Pile)

Max Activating Token Index: 52

Text #14

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1116. Min Act: 0.0000

Data Index: 875823 (The Pile)

Max Activating Token Index: 818

Text #15

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.0806. Min Act: 0.0000

Data Index: 454117 (The Pile)

Max Activating Token Index: 59

Text #16

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1021. Min Act: -0.0000

Data Index: 306859 (The Pile)

Max Activating Token Index: 150

Text #17

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1234. Min Act: -0.0000

Data Index: 1156857 (The Pile)

Max Activating Token Index: 380

Text #18

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1336. Min Act: -0.0000

Data Index: 746693 (The Pile)

Max Activating Token Index: 499

Text #19

Max Range: 1.8391. Min Range: -1.8391

Max Act: 1.1117. Min Act: -0.0000

Data Index: 1740561 (The Pile)

Max Activating Token Index: 912