Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3492 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.7305. Min Act: -0.0000

Data Index: 910333 (The Pile)

Max Activating Token Index: 996

Text #1

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.5365. Min Act: -0.0000

Data Index: 741845 (The Pile)

Max Activating Token Index: 517

Text #2

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.5551. Min Act: -0.0000

Data Index: 780883 (The Pile)

Max Activating Token Index: 77

Text #3

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.5571. Min Act: -0.0000

Data Index: 532951 (The Pile)

Max Activating Token Index: 142

Text #4

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.5974. Min Act: -0.0000

Data Index: 701838 (The Pile)

Max Activating Token Index: 641

Text #5

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.2408. Min Act: -0.0000

Data Index: 1942106 (The Pile)

Max Activating Token Index: 793

Text #6

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3892. Min Act: -0.0000

Data Index: 1548434 (The Pile)

Max Activating Token Index: 283

Text #7

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.5124. Min Act: -0.0000

Data Index: 331861 (The Pile)

Max Activating Token Index: 825

Text #8

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3339. Min Act: -0.0000

Data Index: 538777 (The Pile)

Max Activating Token Index: 387

Text #9

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3898. Min Act: -0.0000

Data Index: 809261 (The Pile)

Max Activating Token Index: 989

Text #10

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3080. Min Act: -0.0000

Data Index: 1194229 (The Pile)

Max Activating Token Index: 848

Text #11

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3707. Min Act: -0.0000

Data Index: 1546957 (The Pile)

Max Activating Token Index: 530

Text #12

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.2286. Min Act: -0.0000

Data Index: 669912 (The Pile)

Max Activating Token Index: 578

Text #13

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3678. Min Act: -0.0000

Data Index: 1539107 (The Pile)

Max Activating Token Index: 262

Text #14

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.4467. Min Act: -0.0000

Data Index: 218988 (The Pile)

Max Activating Token Index: 50

Text #15

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.2938. Min Act: -0.0000

Data Index: 1531756 (The Pile)

Max Activating Token Index: 905

Text #16

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.1885. Min Act: -0.0000

Data Index: 1355957 (The Pile)

Max Activating Token Index: 747

Text #17

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3285. Min Act: -0.0000

Data Index: 1455618 (The Pile)

Max Activating Token Index: 45

Text #18

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3273. Min Act: -0.0000

Data Index: 299639 (The Pile)

Max Activating Token Index: 621

Text #19

Max Range: 1.7305. Min Range: -1.7305

Max Act: 1.3077. Min Act: -0.0000

Data Index: 808035 (The Pile)

Max Activating Token Index: 440