Soaps, of various types, are sodium or potassium salts of fatty acids. They are soluble in water, and good cleaning agents. Other metal ions can replace the sodium and potassium, these are still metal salts of fatty acids, but many aren't soluble. Aluminum "soaps" -- the aluminum salts of fatty acids a very sticky and gloopy -- its mixed with gasoline to make napalm. So soap is a poor choice as a surfactant in polishing aluminum. Dawn is a detergent, its not a metal salt of anything, and is likely not as reactive to aluminum.
I don't know how fast bulk aluminum reacts with soft soaps, and I don't know whats wrong with other detergents that are likely also detergents like Dawn. So I can't fix all the problems you're having. But in a practical way, you should just use Dawn. Or maybe purchase non-ionic or anionic laboratory grade detergents.