Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4272 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.1540. Min Range: -3.1540

Max Act: 3.1540. Min Act: -0.0000

Data Index: 511320 (The Pile)

Max Activating Token Index: 631

Text #1

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.6335. Min Act: 0.0000

Data Index: 403544 (The Pile)

Max Activating Token Index: 337

Text #2

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.6739. Min Act: -0.0000

Data Index: 1848254 (The Pile)

Max Activating Token Index: 623

Text #3

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.1254. Min Act: -0.0000

Data Index: 17148 (The Pile)

Max Activating Token Index: 496

Text #4

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.2868. Min Act: -0.0000

Data Index: 1385123 (The Pile)

Max Activating Token Index: 537

Text #5

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.3690. Min Act: -0.0000

Data Index: 1511716 (The Pile)

Max Activating Token Index: 1016

Text #6

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.1579. Min Act: 0.0000

Data Index: 455053 (The Pile)

Max Activating Token Index: 318

Text #7

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.0514. Min Act: -0.0000

Data Index: 806551 (The Pile)

Max Activating Token Index: 649

Text #8

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.8353. Min Act: -0.0000

Data Index: 1325242 (The Pile)

Max Activating Token Index: 875

Text #9

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.1140. Min Act: -0.0000

Data Index: 791713 (The Pile)

Max Activating Token Index: 1018

Text #10

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9758. Min Act: 0.0000

Data Index: 372548 (The Pile)

Max Activating Token Index: 1003

Text #11

Max Range: 3.1540. Min Range: -3.1540

Max Act: 2.1891. Min Act: -0.0000

Data Index: 1340316 (The Pile)

Max Activating Token Index: 671

Text #12

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9707. Min Act: -0.0000

Data Index: 398919 (The Pile)

Max Activating Token Index: 804

Text #13

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.8552. Min Act: 0.0000

Data Index: 1058556 (The Pile)

Max Activating Token Index: 103

Text #14

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9128. Min Act: -0.0000

Data Index: 894049 (The Pile)

Max Activating Token Index: 177

Text #15

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9720. Min Act: -0.0000

Data Index: 1530549 (The Pile)

Max Activating Token Index: 439

Text #16

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9474. Min Act: -0.0000

Data Index: 1999021 (The Pile)

Max Activating Token Index: 696

Text #17

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9399. Min Act: -0.0000

Data Index: 977709 (The Pile)

Max Activating Token Index: 681

Text #18

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9353. Min Act: -0.0000

Data Index: 788313 (The Pile)

Max Activating Token Index: 754

Text #19

Max Range: 3.1540. Min Range: -3.1540

Max Act: 1.9139. Min Act: 0.0000

Data Index: 228429 (The Pile)

Max Activating Token Index: 488