Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4619 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.7318. Min Act: -0.0000

Data Index: 835970 (The Pile)

Max Activating Token Index: 293

Text #1

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.1029. Min Act: -0.0000

Data Index: 143752 (The Pile)

Max Activating Token Index: 451

Text #2

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0757. Min Act: -0.0000

Data Index: 68035 (The Pile)

Max Activating Token Index: 733

Text #3

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0339. Min Act: -0.0000

Data Index: 1011657 (The Pile)

Max Activating Token Index: 527

Text #4

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.1669. Min Act: -0.0000

Data Index: 565345 (The Pile)

Max Activating Token Index: 164

Text #5

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0472. Min Act: -0.0000

Data Index: 739934 (The Pile)

Max Activating Token Index: 677

Text #6

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0101. Min Act: -0.0000

Data Index: 64185 (The Pile)

Max Activating Token Index: 497

Text #7

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.2603. Min Act: -0.0000

Data Index: 1223560 (The Pile)

Max Activating Token Index: 58

Text #8

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0735. Min Act: -0.0000

Data Index: 619154 (The Pile)

Max Activating Token Index: 674

Text #9

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.6623. Min Act: -0.0000

Data Index: 1298027 (The Pile)

Max Activating Token Index: 768

Text #10

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0434. Min Act: -0.0000

Data Index: 215009 (The Pile)

Max Activating Token Index: 65

Text #11

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0028. Min Act: -0.0000

Data Index: 998687 (The Pile)

Max Activating Token Index: 491

Text #12

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.9921. Min Act: -0.0000

Data Index: 421119 (The Pile)

Max Activating Token Index: 718

Text #13

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.9631. Min Act: -0.0000

Data Index: 1184745 (The Pile)

Max Activating Token Index: 248

Text #14

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.9615. Min Act: -0.0000

Data Index: 1998568 (The Pile)

Max Activating Token Index: 991

Text #15

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.8271. Min Act: -0.0000

Data Index: 51964 (The Pile)

Max Activating Token Index: 256

Text #16

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.8190. Min Act: -0.0000

Data Index: 1876705 (The Pile)

Max Activating Token Index: 500

Text #17

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.9423. Min Act: -0.0000

Data Index: 1877572 (The Pile)

Max Activating Token Index: 188

Text #18

Max Range: 3.7318. Min Range: -3.7318

Max Act: 2.8095. Min Act: -0.0000

Data Index: 1572724 (The Pile)

Max Activating Token Index: 830

Text #19

Max Range: 3.7318. Min Range: -3.7318

Max Act: 3.0753. Min Act: -0.0000

Data Index: 1313862 (The Pile)

Max Activating Token Index: 60