Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2143 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 3.2568. Min Range: -3.2568

Max Act: 3.1463. Min Act: -0.0000

Data Index: 960963 (The Pile)

Max Activating Token Index: 443

Text #1

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.9130. Min Act: -0.0000

Data Index: 1977920 (The Pile)

Max Activating Token Index: 374

Text #2

Max Range: 3.2568. Min Range: -3.2568

Max Act: 3.2568. Min Act: -0.0000

Data Index: 1205418 (The Pile)

Max Activating Token Index: 566

Text #3

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.7733. Min Act: -0.0000

Data Index: 856834 (The Pile)

Max Activating Token Index: 681

Text #4

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.4966. Min Act: -0.0000

Data Index: 1995847 (The Pile)

Max Activating Token Index: 846

Text #5

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5508. Min Act: -0.0000

Data Index: 502508 (The Pile)

Max Activating Token Index: 115

Text #6

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.7754. Min Act: -0.0000

Data Index: 74739 (The Pile)

Max Activating Token Index: 555

Text #7

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.3755. Min Act: -0.0000

Data Index: 404103 (The Pile)

Max Activating Token Index: 533

Text #8

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.6107. Min Act: -0.0000

Data Index: 1407240 (The Pile)

Max Activating Token Index: 719

Text #9

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5721. Min Act: -0.0000

Data Index: 496732 (The Pile)

Max Activating Token Index: 47

Text #10

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.6980. Min Act: -0.0000

Data Index: 1514864 (The Pile)

Max Activating Token Index: 515

Text #11

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5663. Min Act: -0.0000

Data Index: 839412 (The Pile)

Max Activating Token Index: 837

Text #12

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.3992. Min Act: -0.0000

Data Index: 1812618 (The Pile)

Max Activating Token Index: 415

Text #13

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5462. Min Act: -0.0000

Data Index: 1677998 (The Pile)

Max Activating Token Index: 502

Text #14

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.4815. Min Act: -0.0000

Data Index: 315249 (The Pile)

Max Activating Token Index: 573

Text #15

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5629. Min Act: -0.0000

Data Index: 1228114 (The Pile)

Max Activating Token Index: 208

Text #16

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.4356. Min Act: -0.0000

Data Index: 1900533 (The Pile)

Max Activating Token Index: 731

Text #17

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.3606. Min Act: -0.0000

Data Index: 1335015 (The Pile)

Max Activating Token Index: 935

Text #18

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5532. Min Act: -0.0000

Data Index: 1202981 (The Pile)

Max Activating Token Index: 218

Text #19

Max Range: 3.2568. Min Range: -3.2568

Max Act: 2.5766. Min Act: -0.0000

Data Index: 1377160 (The Pile)

Max Activating Token Index: 218