Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 61 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3341. Min Act: -0.0000

Data Index: 1460079 (The Pile)

Max Activating Token Index: 864

Text #1

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3512. Min Act: -0.0000

Data Index: 1709670 (The Pile)

Max Activating Token Index: 962

Text #2

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3125. Min Act: -0.0000

Data Index: 54453 (The Pile)

Max Activating Token Index: 377

Text #3

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3197. Min Act: -0.0000

Data Index: 1089155 (The Pile)

Max Activating Token Index: 689

Text #4

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3197. Min Act: -0.0000

Data Index: 1547616 (The Pile)

Max Activating Token Index: 559

Text #5

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3235. Min Act: -0.0000

Data Index: 370671 (The Pile)

Max Activating Token Index: 499

Text #6

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2980. Min Act: -0.0000

Data Index: 203683 (The Pile)

Max Activating Token Index: 142

Text #7

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2929. Min Act: -0.0000

Data Index: 324427 (The Pile)

Max Activating Token Index: 954

Text #8

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3120. Min Act: -0.0000

Data Index: 1441748 (The Pile)

Max Activating Token Index: 482

Text #9

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2834. Min Act: -0.0000

Data Index: 705723 (The Pile)

Max Activating Token Index: 860

Text #10

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.3027. Min Act: -0.0000

Data Index: 84703 (The Pile)

Max Activating Token Index: 805

Text #11

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2884. Min Act: -0.0000

Data Index: 1867086 (The Pile)

Max Activating Token Index: 64

Text #12

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2876. Min Act: -0.0000

Data Index: 15149 (The Pile)

Max Activating Token Index: 516

Text #13

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2584. Min Act: -0.0000

Data Index: 26718 (The Pile)

Max Activating Token Index: 785

Text #14

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2952. Min Act: -0.0000

Data Index: 1981716 (The Pile)

Max Activating Token Index: 564

Text #15

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2858. Min Act: -0.0000

Data Index: 676976 (The Pile)

Max Activating Token Index: 616

Text #16

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2725. Min Act: -0.0000

Data Index: 1324601 (The Pile)

Max Activating Token Index: 935

Text #17

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2938. Min Act: -0.0000

Data Index: 776781 (The Pile)

Max Activating Token Index: 689

Text #18

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2921. Min Act: -0.0000

Data Index: 1211182 (The Pile)

Max Activating Token Index: 412

Text #19

Max Range: 0.3512. Min Range: -0.3512

Max Act: 0.2641. Min Act: -0.0000

Data Index: 942621 (The Pile)

Max Activating Token Index: 777