Does the fact an element - as it exists in a nature - is a mixture of isotopes, help?
When we deal with large samples of substance (which is form most practical purposes always) we deal with mixtures, so the mass of a single atom doesn't matter. What matters is the observed mass, one that we can experimentally check by weighing the sample and counting atoms in it (by whatever means).
Unit used, and the way it is defined, is secondary. We just took some mass (that was in some way convenient to define and use, but that's irrelevant) and said "from now on this is the standard mass we will use". It is like with using feet, or meters, or inches, or yards, or miles - it really doesn't matter which you use when comparing distances, as long as you are consistent and express all distances using the same unit.