Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2591 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1360. Min Range: -1.1360

Max Act: 1.1360. Min Act: -0.0000

Data Index: 325705 (The Pile)

Max Activating Token Index: 970

Text #1

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.9185. Min Act: -0.0000

Data Index: 1686773 (The Pile)

Max Activating Token Index: 498

Text #2

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8754. Min Act: -0.0000

Data Index: 229735 (The Pile)

Max Activating Token Index: 239

Text #3

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.9162. Min Act: -0.0000

Data Index: 908623 (The Pile)

Max Activating Token Index: 286

Text #4

Max Range: 1.1360. Min Range: -1.1360

Max Act: 1.0919. Min Act: -0.0000

Data Index: 1454389 (The Pile)

Max Activating Token Index: 527

Text #5

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8389. Min Act: -0.0000

Data Index: 67176 (The Pile)

Max Activating Token Index: 875

Text #6

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8791. Min Act: -0.0000

Data Index: 32151 (The Pile)

Max Activating Token Index: 666

Text #7

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8027. Min Act: -0.0000

Data Index: 961775 (The Pile)

Max Activating Token Index: 209

Text #8

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8633. Min Act: -0.0000

Data Index: 1160716 (The Pile)

Max Activating Token Index: 250

Text #9

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8513. Min Act: -0.0000

Data Index: 271502 (The Pile)

Max Activating Token Index: 332

Text #10

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7833. Min Act: -0.0000

Data Index: 220221 (The Pile)

Max Activating Token Index: 806

Text #11

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7683. Min Act: -0.0000

Data Index: 1412762 (The Pile)

Max Activating Token Index: 14

Text #12

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7710. Min Act: -0.0000

Data Index: 1206787 (The Pile)

Max Activating Token Index: 105

Text #13

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.8075. Min Act: -0.0000

Data Index: 576483 (The Pile)

Max Activating Token Index: 331

Text #14

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7364. Min Act: -0.0000

Data Index: 1192267 (The Pile)

Max Activating Token Index: 530

Text #15

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7759. Min Act: -0.0000

Data Index: 1798329 (The Pile)

Max Activating Token Index: 990

Text #16

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.6875. Min Act: -0.0000

Data Index: 236848 (The Pile)

Max Activating Token Index: 475

Text #17

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7322. Min Act: -0.0000

Data Index: 561717 (The Pile)

Max Activating Token Index: 16

Text #18

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.7336. Min Act: -0.0000

Data Index: 885375 (The Pile)

Max Activating Token Index: 650

Text #19

Max Range: 1.1360. Min Range: -1.1360

Max Act: 0.6756. Min Act: -0.0000

Data Index: 817298 (The Pile)

Max Activating Token Index: 648