Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4656 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7399. Min Range: -5.7399

Max Act: 5.7320. Min Act: -0.0000

Data Index: 483084 (The Pile)

Max Activating Token Index: 362

Text #1

Max Range: 5.7399. Min Range: -5.7399

Max Act: 5.7399. Min Act: -0.0000

Data Index: 394613 (The Pile)

Max Activating Token Index: 481

Text #2

Max Range: 5.7399. Min Range: -5.7399

Max Act: 5.5361. Min Act: -0.0000

Data Index: 1858617 (The Pile)

Max Activating Token Index: 987

Text #3

Max Range: 5.7399. Min Range: -5.7399

Max Act: 5.1877. Min Act: -0.0000

Data Index: 1181147 (The Pile)

Max Activating Token Index: 109

Text #4

Max Range: 5.7399. Min Range: -5.7399

Max Act: 4.9185. Min Act: -0.0000

Data Index: 1147433 (The Pile)

Max Activating Token Index: 124

Text #5

Max Range: 5.7399. Min Range: -5.7399

Max Act: 3.5783. Min Act: -0.0000

Data Index: 429542 (The Pile)

Max Activating Token Index: 49

Text #6

Max Range: 5.7399. Min Range: -5.7399

Max Act: 3.5157. Min Act: -0.0000

Data Index: 1156500 (The Pile)

Max Activating Token Index: 914

Text #7

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.7595. Min Act: -0.0000

Data Index: 1629255 (The Pile)

Max Activating Token Index: 106

Text #8

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.6506. Min Act: -0.0000

Data Index: 1791417 (The Pile)

Max Activating Token Index: 759

Text #9

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.7519. Min Act: -0.0000

Data Index: 1902753 (The Pile)

Max Activating Token Index: 80

Text #10

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.5798. Min Act: -0.0000

Data Index: 648163 (The Pile)

Max Activating Token Index: 628

Text #11

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.0268. Min Act: -0.0000

Data Index: 13206 (The Pile)

Max Activating Token Index: 710

Text #12

Max Range: 5.7399. Min Range: -5.7399

Max Act: 2.3324. Min Act: -0.0000

Data Index: 391031 (The Pile)

Max Activating Token Index: 809

Text #13

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.6954. Min Act: -0.0000

Data Index: 558387 (The Pile)

Max Activating Token Index: 879

Text #14

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.9957. Min Act: -0.0000

Data Index: 1840667 (The Pile)

Max Activating Token Index: 836

Text #15

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.8903. Min Act: -0.0000

Data Index: 1013221 (The Pile)

Max Activating Token Index: 317

Text #16

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.9613. Min Act: -0.0000

Data Index: 938826 (The Pile)

Max Activating Token Index: 262

Text #17

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.8816. Min Act: -0.0000

Data Index: 1941316 (The Pile)

Max Activating Token Index: 543

Text #18

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.9684. Min Act: -0.0000

Data Index: 419107 (The Pile)

Max Activating Token Index: 151

Text #19

Max Range: 5.7399. Min Range: -5.7399

Max Act: 1.6584. Min Act: -0.0000

Data Index: 1682059 (The Pile)

Max Activating Token Index: 925