Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4332 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.8043. Min Act: -0.0000

Data Index: 1379887 (The Pile)

Max Activating Token Index: 671

Text #1

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6779. Min Act: -0.0000

Data Index: 1393561 (The Pile)

Max Activating Token Index: 45

Text #2

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6544. Min Act: -0.0000

Data Index: 1648252 (The Pile)

Max Activating Token Index: 817

Text #3

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5843. Min Act: -0.0000

Data Index: 1175792 (The Pile)

Max Activating Token Index: 847

Text #4

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6742. Min Act: -0.0000

Data Index: 59454 (The Pile)

Max Activating Token Index: 786

Text #5

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6250. Min Act: -0.0000

Data Index: 848269 (The Pile)

Max Activating Token Index: 110

Text #6

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6534. Min Act: -0.0000

Data Index: 262139 (The Pile)

Max Activating Token Index: 518

Text #7

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6023. Min Act: -0.0000

Data Index: 863555 (The Pile)

Max Activating Token Index: 466

Text #8

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5757. Min Act: -0.0000

Data Index: 633433 (The Pile)

Max Activating Token Index: 996

Text #9

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5968. Min Act: -0.0000

Data Index: 433747 (The Pile)

Max Activating Token Index: 590

Text #10

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5946. Min Act: -0.0000

Data Index: 476596 (The Pile)

Max Activating Token Index: 581

Text #11

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6375. Min Act: -0.0000

Data Index: 983925 (The Pile)

Max Activating Token Index: 473

Text #12

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5921. Min Act: -0.0000

Data Index: 384644 (The Pile)

Max Activating Token Index: 676

Text #13

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6121. Min Act: -0.0000

Data Index: 9351 (The Pile)

Max Activating Token Index: 117

Text #14

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6538. Min Act: -0.0000

Data Index: 1381369 (The Pile)

Max Activating Token Index: 773

Text #15

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6225. Min Act: -0.0000

Data Index: 335415 (The Pile)

Max Activating Token Index: 684

Text #16

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6045. Min Act: -0.0000

Data Index: 1328163 (The Pile)

Max Activating Token Index: 626

Text #17

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.6144. Min Act: -0.0000

Data Index: 43405 (The Pile)

Max Activating Token Index: 501

Text #18

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5960. Min Act: -0.0000

Data Index: 1532538 (The Pile)

Max Activating Token Index: 732

Text #19

Max Range: 0.8043. Min Range: -0.8043

Max Act: 0.5413. Min Act: -0.0000

Data Index: 1307869 (The Pile)

Max Activating Token Index: 708