Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2264 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.2073. Min Act: -0.0000

Data Index: 1507339 (The Pile)

Max Activating Token Index: 959

Text #1

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1836. Min Act: -0.0000

Data Index: 1851328 (The Pile)

Max Activating Token Index: 537

Text #2

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1892. Min Act: -0.0000

Data Index: 835884 (The Pile)

Max Activating Token Index: 756

Text #3

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1831. Min Act: -0.0000

Data Index: 540362 (The Pile)

Max Activating Token Index: 512

Text #4

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1678. Min Act: -0.0000

Data Index: 942735 (The Pile)

Max Activating Token Index: 391

Text #5

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1742. Min Act: -0.0000

Data Index: 1679032 (The Pile)

Max Activating Token Index: 673

Text #6

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1677. Min Act: -0.0000

Data Index: 335507 (The Pile)

Max Activating Token Index: 400

Text #7

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1733. Min Act: -0.0000

Data Index: 1269625 (The Pile)

Max Activating Token Index: 360

Text #8

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1710. Min Act: -0.0000

Data Index: 138347 (The Pile)

Max Activating Token Index: 496

Text #9

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1729. Min Act: -0.0000

Data Index: 130195 (The Pile)

Max Activating Token Index: 612

Text #10

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1602. Min Act: -0.0000

Data Index: 1489868 (The Pile)

Max Activating Token Index: 667

Text #11

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1582. Min Act: -0.0000

Data Index: 13979 (The Pile)

Max Activating Token Index: 213

Text #12

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1646. Min Act: -0.0000

Data Index: 1979982 (The Pile)

Max Activating Token Index: 543

Text #13

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1793. Min Act: -0.0000

Data Index: 1888153 (The Pile)

Max Activating Token Index: 658

Text #14

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1561. Min Act: -0.0000

Data Index: 317242 (The Pile)

Max Activating Token Index: 132

Text #15

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1465. Min Act: -0.0000

Data Index: 1852103 (The Pile)

Max Activating Token Index: 539

Text #16

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1577. Min Act: -0.0000

Data Index: 319628 (The Pile)

Max Activating Token Index: 471

Text #17

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1467. Min Act: -0.0000

Data Index: 303254 (The Pile)

Max Activating Token Index: 153

Text #18

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1430. Min Act: -0.0000

Data Index: 1909916 (The Pile)

Max Activating Token Index: 229

Text #19

Max Range: 0.2073. Min Range: -0.2073

Max Act: 0.1578. Min Act: -0.0000

Data Index: 1609154 (The Pile)

Max Activating Token Index: 624