Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1456 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.7142. Min Act: -0.0000

Data Index: 998911 (The Pile)

Max Activating Token Index: 959

Text #1

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.5622. Min Act: -0.0000

Data Index: 449860 (The Pile)

Max Activating Token Index: 233

Text #2

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4751. Min Act: -0.0000

Data Index: 1468395 (The Pile)

Max Activating Token Index: 816

Text #3

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4690. Min Act: -0.0000

Data Index: 1861617 (The Pile)

Max Activating Token Index: 475

Text #4

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.5368. Min Act: -0.0000

Data Index: 960682 (The Pile)

Max Activating Token Index: 711

Text #5

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4187. Min Act: -0.0000

Data Index: 369618 (The Pile)

Max Activating Token Index: 611

Text #6

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4874. Min Act: -0.0000

Data Index: 755156 (The Pile)

Max Activating Token Index: 658

Text #7

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.5929. Min Act: -0.0000

Data Index: 147354 (The Pile)

Max Activating Token Index: 738

Text #8

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4339. Min Act: -0.0000

Data Index: 218946 (The Pile)

Max Activating Token Index: 153

Text #9

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4177. Min Act: -0.0000

Data Index: 62340 (The Pile)

Max Activating Token Index: 853

Text #10

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.5968. Min Act: -0.0000

Data Index: 28033 (The Pile)

Max Activating Token Index: 977

Text #11

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.4533. Min Act: -0.0000

Data Index: 1136628 (The Pile)

Max Activating Token Index: 322

Text #12

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.5379. Min Act: -0.0000

Data Index: 877611 (The Pile)

Max Activating Token Index: 707

Text #13

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.3273. Min Act: -0.0000

Data Index: 94967 (The Pile)

Max Activating Token Index: 1019

Text #14

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.3931. Min Act: -0.0000

Data Index: 1905102 (The Pile)

Max Activating Token Index: 183

Text #15

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.3370. Min Act: -0.0000

Data Index: 532483 (The Pile)

Max Activating Token Index: 937

Text #16

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.2743. Min Act: -0.0000

Data Index: 1948497 (The Pile)

Max Activating Token Index: 935

Text #17

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.3307. Min Act: -0.0000

Data Index: 18061 (The Pile)

Max Activating Token Index: 568

Text #18

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.3492. Min Act: -0.0000

Data Index: 1517549 (The Pile)

Max Activating Token Index: 961

Text #19

Max Range: 1.7142. Min Range: -1.7142

Max Act: 1.2838. Min Act: -0.0000

Data Index: 132640 (The Pile)

Max Activating Token Index: 774