Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3397 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.2211. Min Act: -0.0000

Data Index: 1261375 (The Pile)

Max Activating Token Index: 840

Text #1

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.2049. Min Act: -0.0000

Data Index: 963503 (The Pile)

Max Activating Token Index: 900

Text #2

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1755. Min Act: -0.0000

Data Index: 369290 (The Pile)

Max Activating Token Index: 592

Text #3

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1639. Min Act: -0.0000

Data Index: 396202 (The Pile)

Max Activating Token Index: 537

Text #4

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1606. Min Act: -0.0000

Data Index: 995814 (The Pile)

Max Activating Token Index: 453

Text #5

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1613. Min Act: -0.0000

Data Index: 1900293 (The Pile)

Max Activating Token Index: 604

Text #6

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1483. Min Act: -0.0000

Data Index: 1709705 (The Pile)

Max Activating Token Index: 320

Text #7

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1423. Min Act: -0.0000

Data Index: 276321 (The Pile)

Max Activating Token Index: 688

Text #8

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1645. Min Act: -0.0000

Data Index: 1264861 (The Pile)

Max Activating Token Index: 989

Text #9

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1456. Min Act: -0.0000

Data Index: 1470103 (The Pile)

Max Activating Token Index: 205

Text #10

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1469. Min Act: 0.0000

Data Index: 1888863 (The Pile)

Max Activating Token Index: 178

Text #11

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1427. Min Act: -0.0000

Data Index: 1775246 (The Pile)

Max Activating Token Index: 336

Text #12

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1359. Min Act: -0.0000

Data Index: 227513 (The Pile)

Max Activating Token Index: 10

Text #13

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1398. Min Act: -0.0000

Data Index: 850356 (The Pile)

Max Activating Token Index: 873

Text #14

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1394. Min Act: -0.0000

Data Index: 32041 (The Pile)

Max Activating Token Index: 975

Text #15

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1393. Min Act: 0.0000

Data Index: 811098 (The Pile)

Max Activating Token Index: 715

Text #16

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1418. Min Act: -0.0000

Data Index: 599337 (The Pile)

Max Activating Token Index: 698

Text #17

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1415. Min Act: -0.0000

Data Index: 511959 (The Pile)

Max Activating Token Index: 482

Text #18

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1367. Min Act: 0.0000

Data Index: 1738955 (The Pile)

Max Activating Token Index: 181

Text #19

Max Range: 0.2211. Min Range: -0.2211

Max Act: 0.1357. Min Act: 0.0000

Data Index: 918648 (The Pile)

Max Activating Token Index: 49