Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1752 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.7341. Min Act: -0.0000

Data Index: 1683005 (The Pile)

Max Activating Token Index: 977

Text #1

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.7488. Min Act: -0.0000

Data Index: 1825365 (The Pile)

Max Activating Token Index: 78

Text #2

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.7108. Min Act: -0.0000

Data Index: 1330701 (The Pile)

Max Activating Token Index: 12

Text #3

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6795. Min Act: -0.0000

Data Index: 488165 (The Pile)

Max Activating Token Index: 586

Text #4

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6716. Min Act: -0.0000

Data Index: 1665053 (The Pile)

Max Activating Token Index: 1004

Text #5

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6989. Min Act: -0.0000

Data Index: 1214147 (The Pile)

Max Activating Token Index: 177

Text #6

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6830. Min Act: -0.0000

Data Index: 899567 (The Pile)

Max Activating Token Index: 937

Text #7

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6592. Min Act: -0.0000

Data Index: 1829562 (The Pile)

Max Activating Token Index: 489

Text #8

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.7039. Min Act: -0.0000

Data Index: 284621 (The Pile)

Max Activating Token Index: 422

Text #9

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6372. Min Act: -0.0000

Data Index: 123247 (The Pile)

Max Activating Token Index: 514

Text #10

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6610. Min Act: -0.0000

Data Index: 1592406 (The Pile)

Max Activating Token Index: 550

Text #11

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6389. Min Act: -0.0000

Data Index: 673906 (The Pile)

Max Activating Token Index: 820

Text #12

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6403. Min Act: -0.0000

Data Index: 1181807 (The Pile)

Max Activating Token Index: 29

Text #13

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6128. Min Act: -0.0000

Data Index: 841168 (The Pile)

Max Activating Token Index: 235

Text #14

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6523. Min Act: -0.0000

Data Index: 737568 (The Pile)

Max Activating Token Index: 279

Text #15

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6269. Min Act: -0.0000

Data Index: 1439109 (The Pile)

Max Activating Token Index: 531

Text #16

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6052. Min Act: -0.0000

Data Index: 1577103 (The Pile)

Max Activating Token Index: 753

Text #17

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.5641. Min Act: -0.0000

Data Index: 1778298 (The Pile)

Max Activating Token Index: 948

Text #18

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.5541. Min Act: -0.0000

Data Index: 187015 (The Pile)

Max Activating Token Index: 71

Text #19

Max Range: 0.7488. Min Range: -0.7488

Max Act: 0.6332. Min Act: -0.0000

Data Index: 1136261 (The Pile)

Max Activating Token Index: 399