Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1122 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.4947. Min Act: -0.0000

Data Index: 284356 (The Pile)

Max Activating Token Index: 57

Text #1

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.4181. Min Act: -0.0000

Data Index: 510778 (The Pile)

Max Activating Token Index: 27

Text #2

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3946. Min Act: -0.0000

Data Index: 1480373 (The Pile)

Max Activating Token Index: 169

Text #3

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3723. Min Act: -0.0000

Data Index: 1913438 (The Pile)

Max Activating Token Index: 819

Text #4

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3269. Min Act: -0.0000

Data Index: 52987 (The Pile)

Max Activating Token Index: 93

Text #5

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3188. Min Act: -0.0000

Data Index: 108338 (The Pile)

Max Activating Token Index: 109

Text #6

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3278. Min Act: -0.0000

Data Index: 1511786 (The Pile)

Max Activating Token Index: 990

Text #7

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3389. Min Act: -0.0000

Data Index: 1580588 (The Pile)

Max Activating Token Index: 213

Text #8

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2919. Min Act: -0.0000

Data Index: 165548 (The Pile)

Max Activating Token Index: 554

Text #9

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2923. Min Act: -0.0000

Data Index: 1768693 (The Pile)

Max Activating Token Index: 323

Text #10

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3072. Min Act: -0.0000

Data Index: 1623229 (The Pile)

Max Activating Token Index: 462

Text #11

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3084. Min Act: -0.0000

Data Index: 408996 (The Pile)

Max Activating Token Index: 461

Text #12

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3192. Min Act: -0.0000

Data Index: 571285 (The Pile)

Max Activating Token Index: 50

Text #13

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2807. Min Act: -0.0000

Data Index: 1441964 (The Pile)

Max Activating Token Index: 256

Text #14

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2749. Min Act: -0.0000

Data Index: 335358 (The Pile)

Max Activating Token Index: 26

Text #15

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.3227. Min Act: -0.0000

Data Index: 130269 (The Pile)

Max Activating Token Index: 59

Text #16

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2984. Min Act: -0.0000

Data Index: 1784183 (The Pile)

Max Activating Token Index: 641

Text #17

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2589. Min Act: -0.0000

Data Index: 1717548 (The Pile)

Max Activating Token Index: 875

Text #18

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2918. Min Act: -0.0000

Data Index: 1048582 (The Pile)

Max Activating Token Index: 778

Text #19

Max Range: 0.4947. Min Range: -0.4947

Max Act: 0.2941. Min Act: -0.0000

Data Index: 1887850 (The Pile)

Max Activating Token Index: 133