Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3612 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.1743. Min Act: -0.0000

Data Index: 1115537 (The Pile)

Max Activating Token Index: 587

Text #1

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.1768. Min Act: -0.0000

Data Index: 1001769 (The Pile)

Max Activating Token Index: 395

Text #2

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0839. Min Act: -0.0000

Data Index: 275208 (The Pile)

Max Activating Token Index: 150

Text #3

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0992. Min Act: -0.0000

Data Index: 1360856 (The Pile)

Max Activating Token Index: 777

Text #4

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9658. Min Act: -0.0000

Data Index: 155194 (The Pile)

Max Activating Token Index: 715

Text #5

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0944. Min Act: -0.0000

Data Index: 1004489 (The Pile)

Max Activating Token Index: 68

Text #6

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0247. Min Act: -0.0000

Data Index: 323045 (The Pile)

Max Activating Token Index: 39

Text #7

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0541. Min Act: -0.0000

Data Index: 463663 (The Pile)

Max Activating Token Index: 321

Text #8

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0166. Min Act: -0.0000

Data Index: 527429 (The Pile)

Max Activating Token Index: 81

Text #9

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0109. Min Act: -0.0000

Data Index: 1175032 (The Pile)

Max Activating Token Index: 599

Text #10

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9969. Min Act: -0.0000

Data Index: 1523399 (The Pile)

Max Activating Token Index: 315

Text #11

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9551. Min Act: -0.0000

Data Index: 295058 (The Pile)

Max Activating Token Index: 914

Text #12

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9957. Min Act: -0.0000

Data Index: 146498 (The Pile)

Max Activating Token Index: 87

Text #13

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.8506. Min Act: -0.0000

Data Index: 1365232 (The Pile)

Max Activating Token Index: 564

Text #14

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9552. Min Act: -0.0000

Data Index: 100802 (The Pile)

Max Activating Token Index: 618

Text #15

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.1272. Min Act: -0.0000

Data Index: 398472 (The Pile)

Max Activating Token Index: 844

Text #16

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0004. Min Act: -0.0000

Data Index: 1189803 (The Pile)

Max Activating Token Index: 348

Text #17

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0060. Min Act: -0.0000

Data Index: 146575 (The Pile)

Max Activating Token Index: 342

Text #18

Max Range: 1.1768. Min Range: -1.1768

Max Act: 1.0044. Min Act: -0.0000

Data Index: 1842565 (The Pile)

Max Activating Token Index: 737

Text #19

Max Range: 1.1768. Min Range: -1.1768

Max Act: 0.9764. Min Act: -0.0000

Data Index: 230983 (The Pile)

Max Activating Token Index: 300