Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2576 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.9181. Min Act: -0.0000

Data Index: 1124864 (The Pile)

Max Activating Token Index: 702

Text #1

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.8861. Min Act: 0.0000

Data Index: 868118 (The Pile)

Max Activating Token Index: 284

Text #2

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.7030. Min Act: -0.0000

Data Index: 410515 (The Pile)

Max Activating Token Index: 357

Text #3

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5721. Min Act: -0.0000

Data Index: 1603448 (The Pile)

Max Activating Token Index: 905

Text #4

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6236. Min Act: -0.0000

Data Index: 1983577 (The Pile)

Max Activating Token Index: 164

Text #5

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6170. Min Act: -0.0000

Data Index: 463472 (The Pile)

Max Activating Token Index: 474

Text #6

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5700. Min Act: -0.0000

Data Index: 286401 (The Pile)

Max Activating Token Index: 911

Text #7

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5845. Min Act: -0.0000

Data Index: 724543 (The Pile)

Max Activating Token Index: 627

Text #8

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6979. Min Act: -0.0000

Data Index: 1415323 (The Pile)

Max Activating Token Index: 830

Text #9

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6638. Min Act: -0.0000

Data Index: 1763680 (The Pile)

Max Activating Token Index: 394

Text #10

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6556. Min Act: 0.0000

Data Index: 1386419 (The Pile)

Max Activating Token Index: 452

Text #11

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6905. Min Act: 0.0001

Data Index: 1097230 (The Pile)

Max Activating Token Index: 841

Text #12

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6105. Min Act: -0.0000

Data Index: 1868566 (The Pile)

Max Activating Token Index: 931

Text #13

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6134. Min Act: 0.0000

Data Index: 41013 (The Pile)

Max Activating Token Index: 435

Text #14

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5660. Min Act: -0.0000

Data Index: 464250 (The Pile)

Max Activating Token Index: 656

Text #15

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5532. Min Act: -0.0000

Data Index: 1243227 (The Pile)

Max Activating Token Index: 525

Text #16

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6439. Min Act: -0.0000

Data Index: 1205148 (The Pile)

Max Activating Token Index: 678

Text #17

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.6068. Min Act: -0.0000

Data Index: 951245 (The Pile)

Max Activating Token Index: 493

Text #18

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5568. Min Act: -0.0000

Data Index: 1948967 (The Pile)

Max Activating Token Index: 1004

Text #19

Max Range: 0.9181. Min Range: -0.9181

Max Act: 0.5602. Min Act: -0.0000

Data Index: 5752 (The Pile)

Max Activating Token Index: 441