Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3232 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2370. Min Act: -0.0000

Data Index: 50535 (The Pile)

Max Activating Token Index: 479

Text #1

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2478. Min Act: -0.0000

Data Index: 354187 (The Pile)

Max Activating Token Index: 225

Text #2

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2182. Min Act: -0.0000

Data Index: 1415806 (The Pile)

Max Activating Token Index: 793

Text #3

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1899. Min Act: -0.0000

Data Index: 113456 (The Pile)

Max Activating Token Index: 973

Text #4

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2091. Min Act: -0.0000

Data Index: 22877 (The Pile)

Max Activating Token Index: 656

Text #5

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1936. Min Act: -0.0000

Data Index: 11048 (The Pile)

Max Activating Token Index: 668

Text #6

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1749. Min Act: -0.0000

Data Index: 1753379 (The Pile)

Max Activating Token Index: 128

Text #7

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2384. Min Act: -0.0000

Data Index: 109240 (The Pile)

Max Activating Token Index: 177

Text #8

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2074. Min Act: -0.0000

Data Index: 1272795 (The Pile)

Max Activating Token Index: 124

Text #9

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2053. Min Act: -0.0000

Data Index: 657124 (The Pile)

Max Activating Token Index: 413

Text #10

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1971. Min Act: -0.0000

Data Index: 68747 (The Pile)

Max Activating Token Index: 549

Text #11

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1876. Min Act: -0.0000

Data Index: 311780 (The Pile)

Max Activating Token Index: 386

Text #12

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1901. Min Act: -0.0000

Data Index: 1150765 (The Pile)

Max Activating Token Index: 315

Text #13

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.2017. Min Act: -0.0000

Data Index: 974996 (The Pile)

Max Activating Token Index: 267

Text #14

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1909. Min Act: -0.0000

Data Index: 1988400 (The Pile)

Max Activating Token Index: 421

Text #15

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1962. Min Act: -0.0000

Data Index: 1614734 (The Pile)

Max Activating Token Index: 718

Text #16

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1871. Min Act: -0.0000

Data Index: 1023515 (The Pile)

Max Activating Token Index: 312

Text #17

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1672. Min Act: -0.0000

Data Index: 1864030 (The Pile)

Max Activating Token Index: 298

Text #18

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1971. Min Act: -0.0000

Data Index: 1644014 (The Pile)

Max Activating Token Index: 375

Text #19

Max Range: 0.2478. Min Range: -0.2478

Max Act: 0.1633. Min Act: -0.0000

Data Index: 1622789 (The Pile)

Max Activating Token Index: 754