Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2691 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.4066. Min Act: -0.0000

Data Index: 596091 (The Pile)

Max Activating Token Index: 577

Text #1

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.1946. Min Act: -0.0000

Data Index: 1037851 (The Pile)

Max Activating Token Index: 452

Text #2

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.2298. Min Act: -0.0000

Data Index: 114325 (The Pile)

Max Activating Token Index: 766

Text #3

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.2100. Min Act: -0.0000

Data Index: 1889745 (The Pile)

Max Activating Token Index: 513

Text #4

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.1106. Min Act: -0.0000

Data Index: 1144947 (The Pile)

Max Activating Token Index: 619

Text #5

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.1153. Min Act: -0.0000

Data Index: 897524 (The Pile)

Max Activating Token Index: 435

Text #6

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.1056. Min Act: -0.0000

Data Index: 203455 (The Pile)

Max Activating Token Index: 124

Text #7

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.0682. Min Act: -0.0000

Data Index: 281936 (The Pile)

Max Activating Token Index: 626

Text #8

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.0469. Min Act: -0.0000

Data Index: 1673859 (The Pile)

Max Activating Token Index: 49

Text #9

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.0337. Min Act: -0.0000

Data Index: 1951831 (The Pile)

Max Activating Token Index: 185

Text #10

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9754. Min Act: -0.0000

Data Index: 839776 (The Pile)

Max Activating Token Index: 822

Text #11

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9524. Min Act: -0.0000

Data Index: 710025 (The Pile)

Max Activating Token Index: 736

Text #12

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9556. Min Act: -0.0000

Data Index: 1338746 (The Pile)

Max Activating Token Index: 510

Text #13

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.8913. Min Act: -0.0000

Data Index: 620953 (The Pile)

Max Activating Token Index: 505

Text #14

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.8856. Min Act: -0.0000

Data Index: 748877 (The Pile)

Max Activating Token Index: 390

Text #15

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9369. Min Act: -0.0000

Data Index: 1244847 (The Pile)

Max Activating Token Index: 845

Text #16

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9361. Min Act: -0.0000

Data Index: 1953523 (The Pile)

Max Activating Token Index: 589

Text #17

Max Range: 1.4066. Min Range: -1.4066

Max Act: 1.0611. Min Act: -0.0000

Data Index: 1904104 (The Pile)

Max Activating Token Index: 181

Text #18

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.9134. Min Act: -0.0000

Data Index: 1624383 (The Pile)

Max Activating Token Index: 135

Text #19

Max Range: 1.4066. Min Range: -1.4066

Max Act: 0.8593. Min Act: -0.0000

Data Index: 1107725 (The Pile)

Max Activating Token Index: 832