Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1643 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.7977. Min Act: -0.0000

Data Index: 297092 (The Pile)

Max Activating Token Index: 947

Text #1

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.7782. Min Act: -0.0000

Data Index: 1489142 (The Pile)

Max Activating Token Index: 704

Text #2

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.4819. Min Act: -0.0000

Data Index: 1331025 (The Pile)

Max Activating Token Index: 796

Text #3

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.4165. Min Act: -0.0000

Data Index: 1271573 (The Pile)

Max Activating Token Index: 765

Text #4

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.3812. Min Act: -0.0000

Data Index: 624563 (The Pile)

Max Activating Token Index: 648

Text #5

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.1895. Min Act: -0.0000

Data Index: 1031972 (The Pile)

Max Activating Token Index: 415

Text #6

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.5466. Min Act: -0.0000

Data Index: 513059 (The Pile)

Max Activating Token Index: 544

Text #7

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.0367. Min Act: -0.0000

Data Index: 1243367 (The Pile)

Max Activating Token Index: 1013

Text #8

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.0025. Min Act: -0.0000

Data Index: 1626715 (The Pile)

Max Activating Token Index: 501

Text #9

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.4192. Min Act: -0.0000

Data Index: 713947 (The Pile)

Max Activating Token Index: 535

Text #10

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.1991. Min Act: -0.0000

Data Index: 860333 (The Pile)

Max Activating Token Index: 875

Text #11

Max Range: 5.7977. Min Range: -5.7977

Max Act: 4.9832. Min Act: -0.0000

Data Index: 569288 (The Pile)

Max Activating Token Index: 974

Text #12

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.1710. Min Act: -0.0000

Data Index: 483590 (The Pile)

Max Activating Token Index: 607

Text #13

Max Range: 5.7977. Min Range: -5.7977

Max Act: 4.9551. Min Act: -0.0000

Data Index: 1886274 (The Pile)

Max Activating Token Index: 822

Text #14

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.0950. Min Act: -0.0000

Data Index: 1902033 (The Pile)

Max Activating Token Index: 433

Text #15

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.0829. Min Act: -0.0000

Data Index: 1278578 (The Pile)

Max Activating Token Index: 508

Text #16

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.0941. Min Act: -0.0000

Data Index: 779626 (The Pile)

Max Activating Token Index: 232

Text #17

Max Range: 5.7977. Min Range: -5.7977

Max Act: 5.2143. Min Act: -0.0000

Data Index: 1006483 (The Pile)

Max Activating Token Index: 869

Text #18

Max Range: 5.7977. Min Range: -5.7977

Max Act: 4.7769. Min Act: -0.0000

Data Index: 1964059 (The Pile)

Max Activating Token Index: 420

Text #19

Max Range: 5.7977. Min Range: -5.7977

Max Act: 4.9089. Min Act: -0.0000

Data Index: 1908186 (The Pile)

Max Activating Token Index: 810