Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2820 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3919. Min Act: -0.0000

Data Index: 1995195 (The Pile)

Max Activating Token Index: 811

Text #1

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3503. Min Act: -0.0000

Data Index: 697444 (The Pile)

Max Activating Token Index: 403

Text #2

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3782. Min Act: -0.0000

Data Index: 1687814 (The Pile)

Max Activating Token Index: 930

Text #3

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3754. Min Act: -0.0000

Data Index: 258091 (The Pile)

Max Activating Token Index: 93

Text #4

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3573. Min Act: -0.0000

Data Index: 1584432 (The Pile)

Max Activating Token Index: 143

Text #5

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3302. Min Act: -0.0000

Data Index: 439245 (The Pile)

Max Activating Token Index: 365

Text #6

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3083. Min Act: -0.0000

Data Index: 617501 (The Pile)

Max Activating Token Index: 549

Text #7

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2881. Min Act: -0.0000

Data Index: 1801943 (The Pile)

Max Activating Token Index: 328

Text #8

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3098. Min Act: -0.0000

Data Index: 423503 (The Pile)

Max Activating Token Index: 138

Text #9

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2962. Min Act: -0.0000

Data Index: 306481 (The Pile)

Max Activating Token Index: 569

Text #10

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2864. Min Act: -0.0000

Data Index: 1591191 (The Pile)

Max Activating Token Index: 87

Text #11

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2955. Min Act: -0.0000

Data Index: 1409619 (The Pile)

Max Activating Token Index: 249

Text #12

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2797. Min Act: -0.0000

Data Index: 103774 (The Pile)

Max Activating Token Index: 1016

Text #13

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3065. Min Act: -0.0000

Data Index: 883475 (The Pile)

Max Activating Token Index: 989

Text #14

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2922. Min Act: -0.0000

Data Index: 1889603 (The Pile)

Max Activating Token Index: 264

Text #15

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.3008. Min Act: -0.0000

Data Index: 1409514 (The Pile)

Max Activating Token Index: 171

Text #16

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2898. Min Act: -0.0000

Data Index: 1982779 (The Pile)

Max Activating Token Index: 572

Text #17

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2552. Min Act: -0.0000

Data Index: 1901750 (The Pile)

Max Activating Token Index: 933

Text #18

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2686. Min Act: -0.0000

Data Index: 162947 (The Pile)

Max Activating Token Index: 546

Text #19

Max Range: 0.3919. Min Range: -0.3919

Max Act: 0.2773. Min Act: -0.0000

Data Index: 1826550 (The Pile)

Max Activating Token Index: 208