Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2994 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.7545. Min Act: -0.0000

Data Index: 241169 (The Pile)

Max Activating Token Index: 215

Text #1

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.8387. Min Act: -0.0000

Data Index: 640274 (The Pile)

Max Activating Token Index: 801

Text #2

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.7758. Min Act: -0.0000

Data Index: 1425282 (The Pile)

Max Activating Token Index: 101

Text #3

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.7429. Min Act: -0.0000

Data Index: 1101321 (The Pile)

Max Activating Token Index: 11

Text #4

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.7290. Min Act: -0.0000

Data Index: 1043763 (The Pile)

Max Activating Token Index: 915

Text #5

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.7532. Min Act: -0.0000

Data Index: 12452 (The Pile)

Max Activating Token Index: 1015

Text #6

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.6333. Min Act: -0.0000

Data Index: 143570 (The Pile)

Max Activating Token Index: 954

Text #7

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5613. Min Act: -0.0000

Data Index: 613566 (The Pile)

Max Activating Token Index: 1001

Text #8

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5893. Min Act: -0.0000

Data Index: 763556 (The Pile)

Max Activating Token Index: 210

Text #9

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5792. Min Act: -0.0000

Data Index: 1765778 (The Pile)

Max Activating Token Index: 345

Text #10

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5754. Min Act: -0.0000

Data Index: 112009 (The Pile)

Max Activating Token Index: 599

Text #11

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5561. Min Act: -0.0000

Data Index: 475873 (The Pile)

Max Activating Token Index: 205

Text #12

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5330. Min Act: -0.0000

Data Index: 1835835 (The Pile)

Max Activating Token Index: 188

Text #13

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.4428. Min Act: -0.0000

Data Index: 1571193 (The Pile)

Max Activating Token Index: 147

Text #14

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.4940. Min Act: -0.0000

Data Index: 1492785 (The Pile)

Max Activating Token Index: 611

Text #15

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.4765. Min Act: -0.0000

Data Index: 57466 (The Pile)

Max Activating Token Index: 408

Text #16

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.3898. Min Act: -0.0000

Data Index: 1508854 (The Pile)

Max Activating Token Index: 951

Text #17

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.4740. Min Act: -0.0000

Data Index: 271094 (The Pile)

Max Activating Token Index: 898

Text #18

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5035. Min Act: -0.0000

Data Index: 1120544 (The Pile)

Max Activating Token Index: 303

Text #19

Max Range: 1.8387. Min Range: -1.8387

Max Act: 1.5426. Min Act: -0.0000

Data Index: 164448 (The Pile)

Max Activating Token Index: 581