Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4557 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.4807. Min Act: -0.0000

Data Index: 936437 (The Pile)

Max Activating Token Index: 13

Text #1

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.2823. Min Act: -0.0000

Data Index: 297259 (The Pile)

Max Activating Token Index: 121

Text #2

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.2873. Min Act: -0.0000

Data Index: 1254579 (The Pile)

Max Activating Token Index: 533

Text #3

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.2435. Min Act: -0.0000

Data Index: 1822290 (The Pile)

Max Activating Token Index: 967

Text #4

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.2892. Min Act: -0.0000

Data Index: 330184 (The Pile)

Max Activating Token Index: 53

Text #5

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.1488. Min Act: -0.0000

Data Index: 1150984 (The Pile)

Max Activating Token Index: 161

Text #6

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.1950. Min Act: -0.0000

Data Index: 856582 (The Pile)

Max Activating Token Index: 340

Text #7

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.2230. Min Act: -0.0000

Data Index: 690025 (The Pile)

Max Activating Token Index: 905

Text #8

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.0988. Min Act: -0.0000

Data Index: 497435 (The Pile)

Max Activating Token Index: 234

Text #9

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.0248. Min Act: -0.0000

Data Index: 644891 (The Pile)

Max Activating Token Index: 515

Text #10

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.0027. Min Act: -0.0000

Data Index: 12529 (The Pile)

Max Activating Token Index: 420

Text #11

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9994. Min Act: -0.0000

Data Index: 1297851 (The Pile)

Max Activating Token Index: 454

Text #12

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9964. Min Act: -0.0000

Data Index: 1040861 (The Pile)

Max Activating Token Index: 835

Text #13

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.0667. Min Act: -0.0000

Data Index: 1368751 (The Pile)

Max Activating Token Index: 1016

Text #14

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9125. Min Act: -0.0000

Data Index: 1094848 (The Pile)

Max Activating Token Index: 831

Text #15

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9969. Min Act: -0.0000

Data Index: 678861 (The Pile)

Max Activating Token Index: 868

Text #16

Max Range: 1.4807. Min Range: -1.4807

Max Act: 1.0294. Min Act: -0.0000

Data Index: 1087537 (The Pile)

Max Activating Token Index: 478

Text #17

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9268. Min Act: -0.0000

Data Index: 1206942 (The Pile)

Max Activating Token Index: 716

Text #18

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9924. Min Act: -0.0000

Data Index: 1511391 (The Pile)

Max Activating Token Index: 779

Text #19

Max Range: 1.4807. Min Range: -1.4807

Max Act: 0.9941. Min Act: -0.0000

Data Index: 441657 (The Pile)

Max Activating Token Index: 548