Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 145 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.7633. Min Act: -0.0000

Data Index: 669607 (The Pile)

Max Activating Token Index: 568

Text #1

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.4766. Min Act: -0.0000

Data Index: 699543 (The Pile)

Max Activating Token Index: 137

Text #2

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.7477. Min Act: -0.0000

Data Index: 351550 (The Pile)

Max Activating Token Index: 402

Text #3

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.5377. Min Act: 0.0000

Data Index: 1591449 (The Pile)

Max Activating Token Index: 825

Text #4

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.5337. Min Act: -0.0000

Data Index: 1785628 (The Pile)

Max Activating Token Index: 955

Text #5

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.6194. Min Act: -0.0000

Data Index: 521743 (The Pile)

Max Activating Token Index: 321

Text #6

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.4304. Min Act: -0.0000

Data Index: 62906 (The Pile)

Max Activating Token Index: 183

Text #7

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.2413. Min Act: -0.0000

Data Index: 1830831 (The Pile)

Max Activating Token Index: 908

Text #8

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.1782. Min Act: -0.0000

Data Index: 763988 (The Pile)

Max Activating Token Index: 843

Text #9

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.0897. Min Act: -0.0000

Data Index: 179391 (The Pile)

Max Activating Token Index: 578

Text #10

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.0949. Min Act: -0.0000

Data Index: 1421775 (The Pile)

Max Activating Token Index: 620

Text #11

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.8744. Min Act: -0.0000

Data Index: 1677014 (The Pile)

Max Activating Token Index: 845

Text #12

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.0003. Min Act: 0.0000

Data Index: 532912 (The Pile)

Max Activating Token Index: 883

Text #13

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.9750. Min Act: -0.0000

Data Index: 588579 (The Pile)

Max Activating Token Index: 756

Text #14

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.9575. Min Act: -0.0000

Data Index: 1175321 (The Pile)

Max Activating Token Index: 719

Text #15

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.9175. Min Act: 0.0000

Data Index: 1175786 (The Pile)

Max Activating Token Index: 450

Text #16

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.8507. Min Act: -0.0000

Data Index: 1837465 (The Pile)

Max Activating Token Index: 78

Text #17

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.8522. Min Act: -0.0000

Data Index: 1329404 (The Pile)

Max Activating Token Index: 283

Text #18

Max Range: 3.7633. Min Range: -3.7633

Max Act: 2.8791. Min Act: 0.0000

Data Index: 533942 (The Pile)

Max Activating Token Index: 519

Text #19

Max Range: 3.7633. Min Range: -3.7633

Max Act: 3.0165. Min Act: -0.0000

Data Index: 619297 (The Pile)

Max Activating Token Index: 687