Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2565 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.7451. Min Act: -0.0000

Data Index: 1876579 (The Pile)

Max Activating Token Index: 708

Text #1

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.3873. Min Act: -0.0000

Data Index: 1720365 (The Pile)

Max Activating Token Index: 455

Text #2

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.8663. Min Act: -0.0000

Data Index: 1190620 (The Pile)

Max Activating Token Index: 931

Text #3

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.2214. Min Act: -0.0000

Data Index: 215117 (The Pile)

Max Activating Token Index: 663

Text #4

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.1737. Min Act: -0.0000

Data Index: 1282342 (The Pile)

Max Activating Token Index: 946

Text #5

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.3357. Min Act: -0.0000

Data Index: 1044364 (The Pile)

Max Activating Token Index: 533

Text #6

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.2008. Min Act: -0.0000

Data Index: 1757729 (The Pile)

Max Activating Token Index: 500

Text #7

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.1263. Min Act: -0.0000

Data Index: 1151453 (The Pile)

Max Activating Token Index: 104

Text #8

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.4291. Min Act: -0.0000

Data Index: 1462924 (The Pile)

Max Activating Token Index: 873

Text #9

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.2728. Min Act: -0.0000

Data Index: 571257 (The Pile)

Max Activating Token Index: 371

Text #10

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.0902. Min Act: -0.0000

Data Index: 1053241 (The Pile)

Max Activating Token Index: 494

Text #11

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.2351. Min Act: -0.0000

Data Index: 399987 (The Pile)

Max Activating Token Index: 486

Text #12

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.9040. Min Act: -0.0000

Data Index: 1012979 (The Pile)

Max Activating Token Index: 387

Text #13

Max Range: 3.7451. Min Range: -3.7451

Max Act: 3.0150. Min Act: -0.0000

Data Index: 282061 (The Pile)

Max Activating Token Index: 282

Text #14

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.8577. Min Act: -0.0000

Data Index: 1397497 (The Pile)

Max Activating Token Index: 735

Text #15

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.8018. Min Act: -0.0000

Data Index: 1084467 (The Pile)

Max Activating Token Index: 655

Text #16

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.9472. Min Act: -0.0000

Data Index: 563369 (The Pile)

Max Activating Token Index: 366

Text #17

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.9083. Min Act: -0.0000

Data Index: 1976769 (The Pile)

Max Activating Token Index: 704

Text #18

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.9341. Min Act: -0.0000

Data Index: 868077 (The Pile)

Max Activating Token Index: 370

Text #19

Max Range: 3.7451. Min Range: -3.7451

Max Act: 2.6883. Min Act: -0.0000

Data Index: 1998351 (The Pile)

Max Activating Token Index: 154