Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2960 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.1598. Min Act: -0.0000

Data Index: 1048193 (The Pile)

Max Activating Token Index: 811

Text #1

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.2716. Min Act: -0.0000

Data Index: 564951 (The Pile)

Max Activating Token Index: 567

Text #2

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0506. Min Act: -0.0000

Data Index: 1317348 (The Pile)

Max Activating Token Index: 820

Text #3

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0800. Min Act: -0.0000

Data Index: 1369876 (The Pile)

Max Activating Token Index: 765

Text #4

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.1397. Min Act: -0.0000

Data Index: 1569230 (The Pile)

Max Activating Token Index: 258

Text #5

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0737. Min Act: -0.0000

Data Index: 1176331 (The Pile)

Max Activating Token Index: 303

Text #6

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0699. Min Act: -0.0000

Data Index: 1919498 (The Pile)

Max Activating Token Index: 114

Text #7

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0041. Min Act: -0.0000

Data Index: 1323954 (The Pile)

Max Activating Token Index: 918

Text #8

Max Range: 1.2716. Min Range: -1.2716

Max Act: 0.9961. Min Act: -0.0000

Data Index: 1023617 (The Pile)

Max Activating Token Index: 635

Text #9

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0572. Min Act: -0.0000

Data Index: 1883629 (The Pile)

Max Activating Token Index: 175

Text #10

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0201. Min Act: -0.0000

Data Index: 1842039 (The Pile)

Max Activating Token Index: 453

Text #11

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0254. Min Act: -0.0000

Data Index: 1845395 (The Pile)

Max Activating Token Index: 90

Text #12

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0245. Min Act: -0.0000

Data Index: 1191610 (The Pile)

Max Activating Token Index: 513

Text #13

Max Range: 1.2716. Min Range: -1.2716

Max Act: 0.9837. Min Act: -0.0000

Data Index: 1999810 (The Pile)

Max Activating Token Index: 63

Text #14

Max Range: 1.2716. Min Range: -1.2716

Max Act: 0.9731. Min Act: -0.0000

Data Index: 1519155 (The Pile)

Max Activating Token Index: 815

Text #15

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0379. Min Act: -0.0000

Data Index: 296385 (The Pile)

Max Activating Token Index: 676

Text #16

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0422. Min Act: -0.0000

Data Index: 813005 (The Pile)

Max Activating Token Index: 831

Text #17

Max Range: 1.2716. Min Range: -1.2716

Max Act: 1.0665. Min Act: -0.0000

Data Index: 116550 (The Pile)

Max Activating Token Index: 271

Text #18

Max Range: 1.2716. Min Range: -1.2716

Max Act: 0.9557. Min Act: -0.0000

Data Index: 825946 (The Pile)

Max Activating Token Index: 399

Text #19

Max Range: 1.2716. Min Range: -1.2716

Max Act: 0.9258. Min Act: -0.0000

Data Index: 1909144 (The Pile)

Max Activating Token Index: 72