Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1471 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.4816. Min Act: -0.0000

Data Index: 1009433 (The Pile)

Max Activating Token Index: 541

Text #1

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3834. Min Act: -0.0000

Data Index: 1135590 (The Pile)

Max Activating Token Index: 68

Text #2

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3731. Min Act: -0.0000

Data Index: 941549 (The Pile)

Max Activating Token Index: 161

Text #3

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3941. Min Act: -0.0000

Data Index: 1942734 (The Pile)

Max Activating Token Index: 416

Text #4

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3463. Min Act: -0.0000

Data Index: 254233 (The Pile)

Max Activating Token Index: 36

Text #5

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3008. Min Act: -0.0000

Data Index: 895561 (The Pile)

Max Activating Token Index: 100

Text #6

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3356. Min Act: -0.0000

Data Index: 191191 (The Pile)

Max Activating Token Index: 114

Text #7

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3389. Min Act: -0.0000

Data Index: 1938705 (The Pile)

Max Activating Token Index: 63

Text #8

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3339. Min Act: -0.0000

Data Index: 557945 (The Pile)

Max Activating Token Index: 792

Text #9

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3524. Min Act: -0.0000

Data Index: 1785901 (The Pile)

Max Activating Token Index: 86

Text #10

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3191. Min Act: -0.0000

Data Index: 1586990 (The Pile)

Max Activating Token Index: 247

Text #11

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3247. Min Act: -0.0000

Data Index: 173382 (The Pile)

Max Activating Token Index: 455

Text #12

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3445. Min Act: -0.0000

Data Index: 1959261 (The Pile)

Max Activating Token Index: 979

Text #13

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3194. Min Act: -0.0000

Data Index: 1045291 (The Pile)

Max Activating Token Index: 589

Text #14

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.2952. Min Act: -0.0000

Data Index: 1122077 (The Pile)

Max Activating Token Index: 603

Text #15

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3466. Min Act: -0.0000

Data Index: 944341 (The Pile)

Max Activating Token Index: 539

Text #16

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3070. Min Act: -0.0000

Data Index: 1055210 (The Pile)

Max Activating Token Index: 160

Text #17

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3123. Min Act: -0.0000

Data Index: 1000149 (The Pile)

Max Activating Token Index: 669

Text #18

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.3029. Min Act: -0.0000

Data Index: 304081 (The Pile)

Max Activating Token Index: 81

Text #19

Max Range: 0.4816. Min Range: -0.4816

Max Act: 0.2770. Min Act: -0.0000

Data Index: 660192 (The Pile)

Max Activating Token Index: 728