Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2240 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.2081. Min Act: -0.0000

Data Index: 887226 (The Pile)

Max Activating Token Index: 409

Text #1

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0990. Min Act: -0.0000

Data Index: 538800 (The Pile)

Max Activating Token Index: 817

Text #2

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0118. Min Act: -0.0000

Data Index: 819752 (The Pile)

Max Activating Token Index: 601

Text #3

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0073. Min Act: -0.0000

Data Index: 1729876 (The Pile)

Max Activating Token Index: 514

Text #4

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0567. Min Act: -0.0000

Data Index: 154911 (The Pile)

Max Activating Token Index: 951

Text #5

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0948. Min Act: -0.0000

Data Index: 1861672 (The Pile)

Max Activating Token Index: 589

Text #6

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9665. Min Act: -0.0000

Data Index: 1534292 (The Pile)

Max Activating Token Index: 152

Text #7

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9734. Min Act: -0.0000

Data Index: 74541 (The Pile)

Max Activating Token Index: 124

Text #8

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9249. Min Act: -0.0000

Data Index: 1720682 (The Pile)

Max Activating Token Index: 282

Text #9

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9972. Min Act: -0.0000

Data Index: 476857 (The Pile)

Max Activating Token Index: 844

Text #10

Max Range: 1.2081. Min Range: -1.2081

Max Act: 1.0831. Min Act: -0.0000

Data Index: 1489807 (The Pile)

Max Activating Token Index: 860

Text #11

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8464. Min Act: -0.0000

Data Index: 1812916 (The Pile)

Max Activating Token Index: 258

Text #12

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9625. Min Act: -0.0000

Data Index: 1012506 (The Pile)

Max Activating Token Index: 838

Text #13

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8950. Min Act: -0.0000

Data Index: 616977 (The Pile)

Max Activating Token Index: 634

Text #14

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8684. Min Act: -0.0000

Data Index: 1369005 (The Pile)

Max Activating Token Index: 369

Text #15

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8624. Min Act: -0.0000

Data Index: 1639268 (The Pile)

Max Activating Token Index: 591

Text #16

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.9155. Min Act: -0.0000

Data Index: 436673 (The Pile)

Max Activating Token Index: 398

Text #17

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8916. Min Act: -0.0000

Data Index: 481849 (The Pile)

Max Activating Token Index: 163

Text #18

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8680. Min Act: -0.0000

Data Index: 1294297 (The Pile)

Max Activating Token Index: 380

Text #19

Max Range: 1.2081. Min Range: -1.2081

Max Act: 0.8912. Min Act: -0.0000

Data Index: 1464831 (The Pile)

Max Activating Token Index: 59