Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2491 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.7089. Min Range: -5.7089

Max Act: 5.7089. Min Act: -0.0000

Data Index: 1917579 (The Pile)

Max Activating Token Index: 609

Text #1

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.2467. Min Act: -0.0000

Data Index: 795970 (The Pile)

Max Activating Token Index: 532

Text #2

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.4101. Min Act: -0.0000

Data Index: 103251 (The Pile)

Max Activating Token Index: 618

Text #3

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.4565. Min Act: -0.0000

Data Index: 1753525 (The Pile)

Max Activating Token Index: 536

Text #4

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.4772. Min Act: -0.0000

Data Index: 1181514 (The Pile)

Max Activating Token Index: 534

Text #5

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.3357. Min Act: -0.0000

Data Index: 1043000 (The Pile)

Max Activating Token Index: 678

Text #6

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.9709. Min Act: -0.0000

Data Index: 1108913 (The Pile)

Max Activating Token Index: 714

Text #7

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.7433. Min Act: -0.0000

Data Index: 672662 (The Pile)

Max Activating Token Index: 797

Text #8

Max Range: 5.7089. Min Range: -5.7089

Max Act: 4.1662. Min Act: -0.0000

Data Index: 1321454 (The Pile)

Max Activating Token Index: 65

Text #9

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.9107. Min Act: -0.0000

Data Index: 204505 (The Pile)

Max Activating Token Index: 221

Text #10

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.7587. Min Act: -0.0000

Data Index: 855549 (The Pile)

Max Activating Token Index: 51

Text #11

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.5528. Min Act: -0.0000

Data Index: 852823 (The Pile)

Max Activating Token Index: 425

Text #12

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.8103. Min Act: -0.0000

Data Index: 1011317 (The Pile)

Max Activating Token Index: 476

Text #13

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.6464. Min Act: -0.0000

Data Index: 584085 (The Pile)

Max Activating Token Index: 433

Text #14

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.6357. Min Act: -0.0000

Data Index: 1555701 (The Pile)

Max Activating Token Index: 442

Text #15

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.2555. Min Act: -0.0000

Data Index: 935822 (The Pile)

Max Activating Token Index: 841

Text #16

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.4071. Min Act: -0.0000

Data Index: 1661504 (The Pile)

Max Activating Token Index: 846

Text #17

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.5877. Min Act: -0.0000

Data Index: 1453283 (The Pile)

Max Activating Token Index: 471

Text #18

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.1890. Min Act: -0.0000

Data Index: 877970 (The Pile)

Max Activating Token Index: 582

Text #19

Max Range: 5.7089. Min Range: -5.7089

Max Act: 3.6159. Min Act: -0.0000

Data Index: 1369075 (The Pile)

Max Activating Token Index: 892