Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2124 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.6334. Min Act: -0.0000

Data Index: 1287535 (The Pile)

Max Activating Token Index: 724

Text #1

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.5374. Min Act: -0.0000

Data Index: 156454 (The Pile)

Max Activating Token Index: 179

Text #2

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3683. Min Act: -0.0000

Data Index: 131133 (The Pile)

Max Activating Token Index: 181

Text #3

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.4262. Min Act: -0.0000

Data Index: 12099 (The Pile)

Max Activating Token Index: 849

Text #4

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.4199. Min Act: -0.0000

Data Index: 1542130 (The Pile)

Max Activating Token Index: 163

Text #5

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3744. Min Act: -0.0000

Data Index: 1702316 (The Pile)

Max Activating Token Index: 12

Text #6

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.4000. Min Act: -0.0000

Data Index: 1736523 (The Pile)

Max Activating Token Index: 440

Text #7

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3504. Min Act: -0.0000

Data Index: 1905705 (The Pile)

Max Activating Token Index: 27

Text #8

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2917. Min Act: -0.0000

Data Index: 90151 (The Pile)

Max Activating Token Index: 796

Text #9

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3676. Min Act: -0.0000

Data Index: 1267903 (The Pile)

Max Activating Token Index: 13

Text #10

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2859. Min Act: -0.0000

Data Index: 1886284 (The Pile)

Max Activating Token Index: 60

Text #11

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.4318. Min Act: -0.0000

Data Index: 1651387 (The Pile)

Max Activating Token Index: 27

Text #12

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2926. Min Act: -0.0000

Data Index: 1597572 (The Pile)

Max Activating Token Index: 66

Text #13

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2188. Min Act: -0.0000

Data Index: 1750327 (The Pile)

Max Activating Token Index: 26

Text #14

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3490. Min Act: -0.0000

Data Index: 1795433 (The Pile)

Max Activating Token Index: 13

Text #15

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2767. Min Act: -0.0000

Data Index: 785441 (The Pile)

Max Activating Token Index: 205

Text #16

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2044. Min Act: -0.0000

Data Index: 645578 (The Pile)

Max Activating Token Index: 41

Text #17

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2333. Min Act: -0.0000

Data Index: 1321603 (The Pile)

Max Activating Token Index: 103

Text #18

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.3139. Min Act: -0.0000

Data Index: 841425 (The Pile)

Max Activating Token Index: 64

Text #19

Max Range: 1.6334. Min Range: -1.6334

Max Act: 1.2646. Min Act: -0.0000

Data Index: 1691364 (The Pile)

Max Activating Token Index: 134