Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3268 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.6447. Min Act: -0.0000

Data Index: 220276 (The Pile)

Max Activating Token Index: 602

Text #1

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.6611. Min Act: -0.0000

Data Index: 30570 (The Pile)

Max Activating Token Index: 178

Text #2

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.5798. Min Act: -0.0000

Data Index: 1834237 (The Pile)

Max Activating Token Index: 676

Text #3

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.5898. Min Act: -0.0000

Data Index: 1122246 (The Pile)

Max Activating Token Index: 728

Text #4

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.5261. Min Act: -0.0000

Data Index: 300995 (The Pile)

Max Activating Token Index: 666

Text #5

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.5338. Min Act: -0.0000

Data Index: 1616706 (The Pile)

Max Activating Token Index: 128

Text #6

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.5381. Min Act: -0.0000

Data Index: 681757 (The Pile)

Max Activating Token Index: 378

Text #7

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4952. Min Act: -0.0000

Data Index: 307943 (The Pile)

Max Activating Token Index: 697

Text #8

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4989. Min Act: -0.0000

Data Index: 93702 (The Pile)

Max Activating Token Index: 293

Text #9

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4793. Min Act: -0.0000

Data Index: 1971763 (The Pile)

Max Activating Token Index: 652

Text #10

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4670. Min Act: -0.0000

Data Index: 1208057 (The Pile)

Max Activating Token Index: 432

Text #11

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4929. Min Act: -0.0000

Data Index: 806922 (The Pile)

Max Activating Token Index: 205

Text #12

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4645. Min Act: -0.0000

Data Index: 918371 (The Pile)

Max Activating Token Index: 403

Text #13

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4627. Min Act: -0.0000

Data Index: 1088181 (The Pile)

Max Activating Token Index: 642

Text #14

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4696. Min Act: -0.0000

Data Index: 1735389 (The Pile)

Max Activating Token Index: 384

Text #15

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4368. Min Act: -0.0000

Data Index: 1783450 (The Pile)

Max Activating Token Index: 586

Text #16

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4470. Min Act: -0.0000

Data Index: 39419 (The Pile)

Max Activating Token Index: 357

Text #17

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4479. Min Act: -0.0000

Data Index: 933838 (The Pile)

Max Activating Token Index: 295

Text #18

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4474. Min Act: -0.0000

Data Index: 932633 (The Pile)

Max Activating Token Index: 124

Text #19

Max Range: 0.6611. Min Range: -0.6611

Max Act: 0.4300. Min Act: -0.0000

Data Index: 1404652 (The Pile)

Max Activating Token Index: 280