Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3937 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1701. Min Range: -3.1701

Max Act: 3.1701. Min Act: -0.0000

Data Index: 1881018 (The Pile)

Max Activating Token Index: 674

Text #1

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.9037. Min Act: -0.0000

Data Index: 286811 (The Pile)

Max Activating Token Index: 914

Text #2

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.9870. Min Act: -0.0000

Data Index: 630471 (The Pile)

Max Activating Token Index: 898

Text #3

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.8903. Min Act: -0.0000

Data Index: 992930 (The Pile)

Max Activating Token Index: 536

Text #4

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.5864. Min Act: -0.0000

Data Index: 590888 (The Pile)

Max Activating Token Index: 470

Text #5

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.8338. Min Act: -0.0000

Data Index: 1461260 (The Pile)

Max Activating Token Index: 383

Text #6

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.7129. Min Act: -0.0000

Data Index: 1409884 (The Pile)

Max Activating Token Index: 628

Text #7

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.4611. Min Act: -0.0000

Data Index: 1246437 (The Pile)

Max Activating Token Index: 694

Text #8

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.7225. Min Act: -0.0000

Data Index: 1168778 (The Pile)

Max Activating Token Index: 1021

Text #9

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.5730. Min Act: -0.0000

Data Index: 1733353 (The Pile)

Max Activating Token Index: 780

Text #10

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.3621. Min Act: -0.0000

Data Index: 1171587 (The Pile)

Max Activating Token Index: 454

Text #11

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.1969. Min Act: -0.0000

Data Index: 154903 (The Pile)

Max Activating Token Index: 951

Text #12

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.9364. Min Act: -0.0000

Data Index: 69761 (The Pile)

Max Activating Token Index: 1021

Text #13

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.1847. Min Act: -0.0000

Data Index: 115866 (The Pile)

Max Activating Token Index: 989

Text #14

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.2242. Min Act: -0.0000

Data Index: 375060 (The Pile)

Max Activating Token Index: 382

Text #15

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.2073. Min Act: -0.0000

Data Index: 1599540 (The Pile)

Max Activating Token Index: 618

Text #16

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.0673. Min Act: -0.0000

Data Index: 385485 (The Pile)

Max Activating Token Index: 227

Text #17

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.2881. Min Act: -0.0000

Data Index: 1322999 (The Pile)

Max Activating Token Index: 321

Text #18

Max Range: 3.1701. Min Range: -3.1701

Max Act: 1.9455. Min Act: -0.0000

Data Index: 209743 (The Pile)

Max Activating Token Index: 598

Text #19

Max Range: 3.1701. Min Range: -3.1701

Max Act: 2.1479. Min Act: -0.0000

Data Index: 1351409 (The Pile)

Max Activating Token Index: 587