Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4807 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0887. Min Range: -1.0887

Max Act: 1.0887. Min Act: -0.0000

Data Index: 1825719 (The Pile)

Max Activating Token Index: 102

Text #1

Max Range: 1.0887. Min Range: -1.0887

Max Act: 1.0301. Min Act: -0.0000

Data Index: 1591414 (The Pile)

Max Activating Token Index: 940

Text #2

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.8828. Min Act: -0.0000

Data Index: 869227 (The Pile)

Max Activating Token Index: 365

Text #3

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.8375. Min Act: -0.0000

Data Index: 286521 (The Pile)

Max Activating Token Index: 496

Text #4

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.8580. Min Act: -0.0000

Data Index: 1343918 (The Pile)

Max Activating Token Index: 900

Text #5

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7833. Min Act: -0.0000

Data Index: 286621 (The Pile)

Max Activating Token Index: 881

Text #6

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.8327. Min Act: -0.0000

Data Index: 973898 (The Pile)

Max Activating Token Index: 264

Text #7

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7594. Min Act: -0.0000

Data Index: 866220 (The Pile)

Max Activating Token Index: 854

Text #8

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7655. Min Act: -0.0000

Data Index: 1114312 (The Pile)

Max Activating Token Index: 932

Text #9

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7881. Min Act: -0.0000

Data Index: 1046748 (The Pile)

Max Activating Token Index: 581

Text #10

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7641. Min Act: -0.0000

Data Index: 717880 (The Pile)

Max Activating Token Index: 702

Text #11

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7614. Min Act: -0.0000

Data Index: 1450032 (The Pile)

Max Activating Token Index: 236

Text #12

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7453. Min Act: -0.0000

Data Index: 1781510 (The Pile)

Max Activating Token Index: 1017

Text #13

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7093. Min Act: -0.0000

Data Index: 993 (The Pile)

Max Activating Token Index: 626

Text #14

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.6993. Min Act: -0.0000

Data Index: 1619328 (The Pile)

Max Activating Token Index: 648

Text #15

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7109. Min Act: -0.0000

Data Index: 1149321 (The Pile)

Max Activating Token Index: 593

Text #16

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7366. Min Act: -0.0000

Data Index: 588516 (The Pile)

Max Activating Token Index: 580

Text #17

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.7069. Min Act: -0.0000

Data Index: 399236 (The Pile)

Max Activating Token Index: 169

Text #18

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.6832. Min Act: -0.0000

Data Index: 1931648 (The Pile)

Max Activating Token Index: 702

Text #19

Max Range: 1.0887. Min Range: -1.0887

Max Act: 0.6992. Min Act: -0.0000

Data Index: 791002 (The Pile)

Max Activating Token Index: 450