Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 958 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.5150. Min Act: -0.0000

Data Index: 1655809 (The Pile)

Max Activating Token Index: 67

Text #1

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.3620. Min Act: -0.0000

Data Index: 1729311 (The Pile)

Max Activating Token Index: 673

Text #2

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.3509. Min Act: -0.0000

Data Index: 1586655 (The Pile)

Max Activating Token Index: 228

Text #3

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2503. Min Act: -0.0000

Data Index: 1831402 (The Pile)

Max Activating Token Index: 873

Text #4

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2743. Min Act: -0.0000

Data Index: 733227 (The Pile)

Max Activating Token Index: 515

Text #5

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2251. Min Act: -0.0000

Data Index: 612875 (The Pile)

Max Activating Token Index: 947

Text #6

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2438. Min Act: -0.0000

Data Index: 570475 (The Pile)

Max Activating Token Index: 18

Text #7

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2355. Min Act: -0.0000

Data Index: 820317 (The Pile)

Max Activating Token Index: 184

Text #8

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2150. Min Act: -0.0000

Data Index: 128769 (The Pile)

Max Activating Token Index: 132

Text #9

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.2464. Min Act: -0.0000

Data Index: 34494 (The Pile)

Max Activating Token Index: 42

Text #10

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1242. Min Act: -0.0000

Data Index: 1140797 (The Pile)

Max Activating Token Index: 270

Text #11

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.0922. Min Act: -0.0000

Data Index: 958348 (The Pile)

Max Activating Token Index: 727

Text #12

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1287. Min Act: -0.0000

Data Index: 304003 (The Pile)

Max Activating Token Index: 105

Text #13

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1211. Min Act: -0.0000

Data Index: 549048 (The Pile)

Max Activating Token Index: 337

Text #14

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1560. Min Act: -0.0000

Data Index: 421134 (The Pile)

Max Activating Token Index: 701

Text #15

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.0798. Min Act: -0.0000

Data Index: 1644536 (The Pile)

Max Activating Token Index: 336

Text #16

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1269. Min Act: -0.0000

Data Index: 1414523 (The Pile)

Max Activating Token Index: 1001

Text #17

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.0875. Min Act: -0.0000

Data Index: 111898 (The Pile)

Max Activating Token Index: 931

Text #18

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.1139. Min Act: -0.0000

Data Index: 142676 (The Pile)

Max Activating Token Index: 969

Text #19

Max Range: 1.5150. Min Range: -1.5150

Max Act: 1.0747. Min Act: -0.0000

Data Index: 1248821 (The Pile)

Max Activating Token Index: 747