Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2545 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8346. Min Act: -0.0000

Data Index: 798070 (The Pile)

Max Activating Token Index: 41

Text #1

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8503. Min Act: -0.0000

Data Index: 1837383 (The Pile)

Max Activating Token Index: 660

Text #2

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8325. Min Act: -0.0000

Data Index: 1559099 (The Pile)

Max Activating Token Index: 103

Text #3

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8352. Min Act: -0.0000

Data Index: 1620911 (The Pile)

Max Activating Token Index: 132

Text #4

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8098. Min Act: -0.0000

Data Index: 797270 (The Pile)

Max Activating Token Index: 26

Text #5

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8001. Min Act: -0.0000

Data Index: 292127 (The Pile)

Max Activating Token Index: 212

Text #6

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7787. Min Act: -0.0000

Data Index: 636686 (The Pile)

Max Activating Token Index: 205

Text #7

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7809. Min Act: -0.0000

Data Index: 1440454 (The Pile)

Max Activating Token Index: 373

Text #8

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.8217. Min Act: -0.0000

Data Index: 1613121 (The Pile)

Max Activating Token Index: 930

Text #9

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7483. Min Act: -0.0000

Data Index: 483283 (The Pile)

Max Activating Token Index: 245

Text #10

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7678. Min Act: -0.0000

Data Index: 1960141 (The Pile)

Max Activating Token Index: 137

Text #11

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7690. Min Act: -0.0000

Data Index: 251408 (The Pile)

Max Activating Token Index: 912

Text #12

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7676. Min Act: -0.0000

Data Index: 1490744 (The Pile)

Max Activating Token Index: 1014

Text #13

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7548. Min Act: -0.0000

Data Index: 33960 (The Pile)

Max Activating Token Index: 735

Text #14

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7314. Min Act: -0.0000

Data Index: 1448049 (The Pile)

Max Activating Token Index: 174

Text #15

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7504. Min Act: -0.0000

Data Index: 987327 (The Pile)

Max Activating Token Index: 203

Text #16

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.6954. Min Act: -0.0000

Data Index: 304673 (The Pile)

Max Activating Token Index: 912

Text #17

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7409. Min Act: -0.0000

Data Index: 526966 (The Pile)

Max Activating Token Index: 654

Text #18

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7646. Min Act: -0.0000

Data Index: 1391278 (The Pile)

Max Activating Token Index: 375

Text #19

Max Range: 0.8503. Min Range: -0.8503

Max Act: 0.7398. Min Act: -0.0000

Data Index: 1300980 (The Pile)

Max Activating Token Index: 809