Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4936 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.9784. Min Act: -0.0000

Data Index: 123171 (The Pile)

Max Activating Token Index: 335

Text #1

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6994. Min Act: -0.0000

Data Index: 832831 (The Pile)

Max Activating Token Index: 863

Text #2

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.7820. Min Act: -0.0000

Data Index: 548138 (The Pile)

Max Activating Token Index: 488

Text #3

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.8836. Min Act: -0.0000

Data Index: 1097938 (The Pile)

Max Activating Token Index: 434

Text #4

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.7345. Min Act: -0.0000

Data Index: 1580049 (The Pile)

Max Activating Token Index: 762

Text #5

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.8275. Min Act: -0.0000

Data Index: 1255723 (The Pile)

Max Activating Token Index: 207

Text #6

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6901. Min Act: -0.0000

Data Index: 1575342 (The Pile)

Max Activating Token Index: 802

Text #7

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.7183. Min Act: -0.0000

Data Index: 688821 (The Pile)

Max Activating Token Index: 9

Text #8

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6666. Min Act: -0.0000

Data Index: 1964678 (The Pile)

Max Activating Token Index: 241

Text #9

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.5784. Min Act: -0.0000

Data Index: 786877 (The Pile)

Max Activating Token Index: 705

Text #10

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6233. Min Act: -0.0000

Data Index: 319438 (The Pile)

Max Activating Token Index: 993

Text #11

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6105. Min Act: -0.0000

Data Index: 288675 (The Pile)

Max Activating Token Index: 307

Text #12

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6037. Min Act: -0.0000

Data Index: 211745 (The Pile)

Max Activating Token Index: 704

Text #13

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6040. Min Act: -0.0000

Data Index: 1964494 (The Pile)

Max Activating Token Index: 251

Text #14

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.5683. Min Act: -0.0000

Data Index: 1977968 (The Pile)

Max Activating Token Index: 750

Text #15

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.5786. Min Act: -0.0000

Data Index: 1191508 (The Pile)

Max Activating Token Index: 407

Text #16

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.8751. Min Act: -0.0000

Data Index: 1560736 (The Pile)

Max Activating Token Index: 944

Text #17

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6428. Min Act: -0.0000

Data Index: 229870 (The Pile)

Max Activating Token Index: 672

Text #18

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.6601. Min Act: -0.0000

Data Index: 1622098 (The Pile)

Max Activating Token Index: 780

Text #19

Max Range: 1.9784. Min Range: -1.9784

Max Act: 1.5452. Min Act: -0.0000

Data Index: 273841 (The Pile)

Max Activating Token Index: 426