Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4657 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.5446. Min Act: -0.0000

Data Index: 1081825 (The Pile)

Max Activating Token Index: 57

Text #1

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3723. Min Act: -0.0000

Data Index: 554869 (The Pile)

Max Activating Token Index: 75

Text #2

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.4046. Min Act: -0.0000

Data Index: 456194 (The Pile)

Max Activating Token Index: 233

Text #3

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.4280. Min Act: -0.0000

Data Index: 1664200 (The Pile)

Max Activating Token Index: 533

Text #4

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3295. Min Act: -0.0000

Data Index: 1610974 (The Pile)

Max Activating Token Index: 32

Text #5

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3628. Min Act: -0.0000

Data Index: 308797 (The Pile)

Max Activating Token Index: 711

Text #6

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3140. Min Act: -0.0000

Data Index: 1479805 (The Pile)

Max Activating Token Index: 960

Text #7

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2496. Min Act: -0.0000

Data Index: 409384 (The Pile)

Max Activating Token Index: 159

Text #8

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3953. Min Act: -0.0000

Data Index: 1471336 (The Pile)

Max Activating Token Index: 588

Text #9

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3240. Min Act: -0.0000

Data Index: 1423595 (The Pile)

Max Activating Token Index: 405

Text #10

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.3079. Min Act: -0.0000

Data Index: 1174628 (The Pile)

Max Activating Token Index: 1017

Text #11

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2995. Min Act: -0.0000

Data Index: 1362851 (The Pile)

Max Activating Token Index: 732

Text #12

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2094. Min Act: -0.0000

Data Index: 1867185 (The Pile)

Max Activating Token Index: 685

Text #13

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2759. Min Act: -0.0000

Data Index: 391500 (The Pile)

Max Activating Token Index: 41

Text #14

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2251. Min Act: -0.0000

Data Index: 742204 (The Pile)

Max Activating Token Index: 1002

Text #15

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.1773. Min Act: -0.0000

Data Index: 694560 (The Pile)

Max Activating Token Index: 105

Text #16

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2108. Min Act: -0.0000

Data Index: 1426729 (The Pile)

Max Activating Token Index: 154

Text #17

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.1908. Min Act: -0.0000

Data Index: 1999545 (The Pile)

Max Activating Token Index: 139

Text #18

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2211. Min Act: -0.0000

Data Index: 1241254 (The Pile)

Max Activating Token Index: 943

Text #19

Max Range: 1.5446. Min Range: -1.5446

Max Act: 1.2582. Min Act: -0.0000

Data Index: 1971360 (The Pile)

Max Activating Token Index: 256