Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3475 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.1939. Min Act: -0.0000

Data Index: 98790 (The Pile)

Max Activating Token Index: 613

Text #1

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.1712. Min Act: -0.0000

Data Index: 1391417 (The Pile)

Max Activating Token Index: 686

Text #2

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.2049. Min Act: -0.0000

Data Index: 372890 (The Pile)

Max Activating Token Index: 107

Text #3

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.1434. Min Act: -0.0000

Data Index: 1858855 (The Pile)

Max Activating Token Index: 135

Text #4

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.1331. Min Act: -0.0000

Data Index: 836107 (The Pile)

Max Activating Token Index: 111

Text #5

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0684. Min Act: -0.0000

Data Index: 1879274 (The Pile)

Max Activating Token Index: 762

Text #6

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0599. Min Act: -0.0000

Data Index: 102186 (The Pile)

Max Activating Token Index: 250

Text #7

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9950. Min Act: -0.0000

Data Index: 1623547 (The Pile)

Max Activating Token Index: 863

Text #8

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0058. Min Act: -0.0000

Data Index: 199850 (The Pile)

Max Activating Token Index: 256

Text #9

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9908. Min Act: -0.0000

Data Index: 954239 (The Pile)

Max Activating Token Index: 986

Text #10

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9809. Min Act: -0.0000

Data Index: 1062028 (The Pile)

Max Activating Token Index: 196

Text #11

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0328. Min Act: -0.0000

Data Index: 197048 (The Pile)

Max Activating Token Index: 601

Text #12

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0496. Min Act: -0.0000

Data Index: 357176 (The Pile)

Max Activating Token Index: 776

Text #13

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0008. Min Act: -0.0000

Data Index: 1325959 (The Pile)

Max Activating Token Index: 398

Text #14

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9946. Min Act: -0.0000

Data Index: 1263323 (The Pile)

Max Activating Token Index: 472

Text #15

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.8536. Min Act: -0.0000

Data Index: 1693137 (The Pile)

Max Activating Token Index: 894

Text #16

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9952. Min Act: -0.0000

Data Index: 1400856 (The Pile)

Max Activating Token Index: 534

Text #17

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.1168. Min Act: -0.0000

Data Index: 1224521 (The Pile)

Max Activating Token Index: 829

Text #18

Max Range: 1.2049. Min Range: -1.2049

Max Act: 0.9880. Min Act: -0.0000

Data Index: 19963 (The Pile)

Max Activating Token Index: 813

Text #19

Max Range: 1.2049. Min Range: -1.2049

Max Act: 1.0472. Min Act: -0.0000

Data Index: 600953 (The Pile)

Max Activating Token Index: 19