Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4553 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.2705. Min Act: -0.0000

Data Index: 311551 (The Pile)

Max Activating Token Index: 516

Text #1

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.1646. Min Act: -0.0000

Data Index: 306394 (The Pile)

Max Activating Token Index: 224

Text #2

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.1038. Min Act: -0.0000

Data Index: 526982 (The Pile)

Max Activating Token Index: 1001

Text #3

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0950. Min Act: -0.0000

Data Index: 701191 (The Pile)

Max Activating Token Index: 320

Text #4

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0927. Min Act: -0.0000

Data Index: 50062 (The Pile)

Max Activating Token Index: 694

Text #5

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0524. Min Act: -0.0000

Data Index: 1848064 (The Pile)

Max Activating Token Index: 521

Text #6

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.1266. Min Act: -0.0000

Data Index: 1629567 (The Pile)

Max Activating Token Index: 616

Text #7

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0554. Min Act: -0.0000

Data Index: 1624789 (The Pile)

Max Activating Token Index: 214

Text #8

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0288. Min Act: -0.0000

Data Index: 477455 (The Pile)

Max Activating Token Index: 1019

Text #9

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0192. Min Act: -0.0000

Data Index: 1766597 (The Pile)

Max Activating Token Index: 958

Text #10

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0081. Min Act: -0.0000

Data Index: 588969 (The Pile)

Max Activating Token Index: 640

Text #11

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.9990. Min Act: -0.0000

Data Index: 696604 (The Pile)

Max Activating Token Index: 602

Text #12

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0373. Min Act: -0.0000

Data Index: 1540348 (The Pile)

Max Activating Token Index: 203

Text #13

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0879. Min Act: -0.0000

Data Index: 1366126 (The Pile)

Max Activating Token Index: 908

Text #14

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.9714. Min Act: -0.0000

Data Index: 1469481 (The Pile)

Max Activating Token Index: 107

Text #15

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.9956. Min Act: -0.0000

Data Index: 296027 (The Pile)

Max Activating Token Index: 584

Text #16

Max Range: 1.2705. Min Range: -1.2705

Max Act: 1.0870. Min Act: -0.0000

Data Index: 1929612 (The Pile)

Max Activating Token Index: 891

Text #17

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.8713. Min Act: -0.0000

Data Index: 17652 (The Pile)

Max Activating Token Index: 895

Text #18

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.9296. Min Act: -0.0000

Data Index: 1061968 (The Pile)

Max Activating Token Index: 589

Text #19

Max Range: 1.2705. Min Range: -1.2705

Max Act: 0.9989. Min Act: -0.0000

Data Index: 1028087 (The Pile)

Max Activating Token Index: 276