Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2432 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.7963. Min Act: -0.0000

Data Index: 160326 (The Pile)

Max Activating Token Index: 458

Text #1

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6790. Min Act: -0.0000

Data Index: 1143524 (The Pile)

Max Activating Token Index: 668

Text #2

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.7109. Min Act: -0.0000

Data Index: 800093 (The Pile)

Max Activating Token Index: 385

Text #3

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6442. Min Act: -0.0000

Data Index: 1692402 (The Pile)

Max Activating Token Index: 742

Text #4

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6561. Min Act: -0.0000

Data Index: 422416 (The Pile)

Max Activating Token Index: 110

Text #5

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6881. Min Act: -0.0000

Data Index: 593078 (The Pile)

Max Activating Token Index: 806

Text #6

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6364. Min Act: -0.0000

Data Index: 1605206 (The Pile)

Max Activating Token Index: 53

Text #7

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6074. Min Act: -0.0000

Data Index: 1325642 (The Pile)

Max Activating Token Index: 180

Text #8

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6333. Min Act: -0.0000

Data Index: 555732 (The Pile)

Max Activating Token Index: 201

Text #9

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5965. Min Act: -0.0000

Data Index: 1221215 (The Pile)

Max Activating Token Index: 385

Text #10

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5325. Min Act: -0.0000

Data Index: 623676 (The Pile)

Max Activating Token Index: 900

Text #11

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5948. Min Act: -0.0000

Data Index: 1877503 (The Pile)

Max Activating Token Index: 431

Text #12

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5420. Min Act: -0.0000

Data Index: 1408996 (The Pile)

Max Activating Token Index: 625

Text #13

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5510. Min Act: -0.0000

Data Index: 1189163 (The Pile)

Max Activating Token Index: 930

Text #14

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.6354. Min Act: -0.0000

Data Index: 1237672 (The Pile)

Max Activating Token Index: 784

Text #15

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5823. Min Act: -0.0000

Data Index: 207067 (The Pile)

Max Activating Token Index: 29

Text #16

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5568. Min Act: -0.0000

Data Index: 277799 (The Pile)

Max Activating Token Index: 253

Text #17

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5918. Min Act: -0.0000

Data Index: 706900 (The Pile)

Max Activating Token Index: 325

Text #18

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5738. Min Act: -0.0000

Data Index: 961645 (The Pile)

Max Activating Token Index: 326

Text #19

Max Range: 0.7963. Min Range: -0.7963

Max Act: 0.5489. Min Act: -0.0000

Data Index: 847489 (The Pile)

Max Activating Token Index: 510