Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 583 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.4286. Min Act: -0.0000

Data Index: 1107519 (The Pile)

Max Activating Token Index: 198

Text #1

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.4454. Min Act: -0.0000

Data Index: 800887 (The Pile)

Max Activating Token Index: 270

Text #2

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.4063. Min Act: -0.0000

Data Index: 819024 (The Pile)

Max Activating Token Index: 362

Text #3

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3902. Min Act: -0.0000

Data Index: 1145726 (The Pile)

Max Activating Token Index: 803

Text #4

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.4006. Min Act: -0.0000

Data Index: 1183270 (The Pile)

Max Activating Token Index: 611

Text #5

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3709. Min Act: -0.0000

Data Index: 863806 (The Pile)

Max Activating Token Index: 635

Text #6

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3620. Min Act: 0.0000

Data Index: 1276088 (The Pile)

Max Activating Token Index: 941

Text #7

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3817. Min Act: -0.0000

Data Index: 572224 (The Pile)

Max Activating Token Index: 380

Text #8

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3744. Min Act: -0.0000

Data Index: 619379 (The Pile)

Max Activating Token Index: 319

Text #9

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3805. Min Act: -0.0000

Data Index: 1475096 (The Pile)

Max Activating Token Index: 446

Text #10

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3594. Min Act: -0.0000

Data Index: 242918 (The Pile)

Max Activating Token Index: 399

Text #11

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3811. Min Act: -0.0000

Data Index: 1732426 (The Pile)

Max Activating Token Index: 755

Text #12

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3616. Min Act: -0.0000

Data Index: 1534125 (The Pile)

Max Activating Token Index: 732

Text #13

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3405. Min Act: -0.0000

Data Index: 780550 (The Pile)

Max Activating Token Index: 578

Text #14

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3714. Min Act: -0.0000

Data Index: 889665 (The Pile)

Max Activating Token Index: 881

Text #15

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3537. Min Act: -0.0000

Data Index: 1921499 (The Pile)

Max Activating Token Index: 759

Text #16

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3421. Min Act: -0.0000

Data Index: 1466369 (The Pile)

Max Activating Token Index: 530

Text #17

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3439. Min Act: -0.0000

Data Index: 696873 (The Pile)

Max Activating Token Index: 529

Text #18

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3365. Min Act: -0.0000

Data Index: 1142465 (The Pile)

Max Activating Token Index: 254

Text #19

Max Range: 0.4454. Min Range: -0.4454

Max Act: 0.3535. Min Act: -0.0000

Data Index: 679051 (The Pile)

Max Activating Token Index: 292