Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 469 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.8461. Min Act: -0.0000

Data Index: 1383644 (The Pile)

Max Activating Token Index: 329

Text #1

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7388. Min Act: -0.0000

Data Index: 773318 (The Pile)

Max Activating Token Index: 921

Text #2

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7381. Min Act: -0.0000

Data Index: 887271 (The Pile)

Max Activating Token Index: 904

Text #3

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7799. Min Act: -0.0000

Data Index: 1499943 (The Pile)

Max Activating Token Index: 679

Text #4

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7139. Min Act: -0.0000

Data Index: 598272 (The Pile)

Max Activating Token Index: 792

Text #5

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6990. Min Act: -0.0000

Data Index: 690917 (The Pile)

Max Activating Token Index: 463

Text #6

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7390. Min Act: -0.0000

Data Index: 1375141 (The Pile)

Max Activating Token Index: 345

Text #7

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7311. Min Act: -0.0000

Data Index: 1041833 (The Pile)

Max Activating Token Index: 649

Text #8

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7378. Min Act: -0.0000

Data Index: 634329 (The Pile)

Max Activating Token Index: 440

Text #9

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7166. Min Act: -0.0000

Data Index: 1633748 (The Pile)

Max Activating Token Index: 597

Text #10

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7164. Min Act: -0.0000

Data Index: 1931015 (The Pile)

Max Activating Token Index: 578

Text #11

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7118. Min Act: -0.0000

Data Index: 890580 (The Pile)

Max Activating Token Index: 517

Text #12

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7298. Min Act: -0.0000

Data Index: 612886 (The Pile)

Max Activating Token Index: 424

Text #13

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6998. Min Act: -0.0000

Data Index: 1250792 (The Pile)

Max Activating Token Index: 480

Text #14

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.7473. Min Act: -0.0000

Data Index: 163009 (The Pile)

Max Activating Token Index: 590

Text #15

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6288. Min Act: -0.0000

Data Index: 199455 (The Pile)

Max Activating Token Index: 419

Text #16

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6678. Min Act: -0.0000

Data Index: 755023 (The Pile)

Max Activating Token Index: 463

Text #17

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.5986. Min Act: -0.0000

Data Index: 1263207 (The Pile)

Max Activating Token Index: 822

Text #18

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6998. Min Act: -0.0000

Data Index: 915116 (The Pile)

Max Activating Token Index: 260

Text #19

Max Range: 0.8461. Min Range: -0.8461

Max Act: 0.6541. Min Act: -0.0000

Data Index: 771109 (The Pile)

Max Activating Token Index: 721