Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 177 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9778. Min Act: -0.0000

Data Index: 414466 (The Pile)

Max Activating Token Index: 731

Text #1

Max Range: 1.0506. Min Range: -1.0506

Max Act: 1.0217. Min Act: -0.0000

Data Index: 1781442 (The Pile)

Max Activating Token Index: 678

Text #2

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9505. Min Act: -0.0000

Data Index: 513513 (The Pile)

Max Activating Token Index: 493

Text #3

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9617. Min Act: -0.0000

Data Index: 1281079 (The Pile)

Max Activating Token Index: 581

Text #4

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9769. Min Act: -0.0000

Data Index: 1668788 (The Pile)

Max Activating Token Index: 387

Text #5

Max Range: 1.0506. Min Range: -1.0506

Max Act: 1.0506. Min Act: -0.0000

Data Index: 1200632 (The Pile)

Max Activating Token Index: 207

Text #6

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9233. Min Act: -0.0000

Data Index: 760717 (The Pile)

Max Activating Token Index: 701

Text #7

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7230. Min Act: -0.0000

Data Index: 459125 (The Pile)

Max Activating Token Index: 130

Text #8

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.8409. Min Act: -0.0000

Data Index: 142730 (The Pile)

Max Activating Token Index: 521

Text #9

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.9510. Min Act: -0.0000

Data Index: 1411755 (The Pile)

Max Activating Token Index: 844

Text #10

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.8644. Min Act: -0.0000

Data Index: 682548 (The Pile)

Max Activating Token Index: 411

Text #11

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.8084. Min Act: -0.0000

Data Index: 243836 (The Pile)

Max Activating Token Index: 328

Text #12

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7829. Min Act: -0.0000

Data Index: 846040 (The Pile)

Max Activating Token Index: 749

Text #13

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7888. Min Act: -0.0000

Data Index: 225140 (The Pile)

Max Activating Token Index: 941

Text #14

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7328. Min Act: -0.0000

Data Index: 102162 (The Pile)

Max Activating Token Index: 929

Text #15

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7271. Min Act: -0.0000

Data Index: 1463447 (The Pile)

Max Activating Token Index: 908

Text #16

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.8154. Min Act: -0.0000

Data Index: 628944 (The Pile)

Max Activating Token Index: 844

Text #17

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.7363. Min Act: -0.0000

Data Index: 37017 (The Pile)

Max Activating Token Index: 145

Text #18

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.6461. Min Act: -0.0000

Data Index: 1495205 (The Pile)

Max Activating Token Index: 72

Text #19

Max Range: 1.0506. Min Range: -1.0506

Max Act: 0.6575. Min Act: -0.0000

Data Index: 1188700 (The Pile)

Max Activating Token Index: 897