For one, it's a matter of relative solubiility.
The other contribution is that crystallization itself is very selective, like solidificarion is, and more so under the proper (meaning slow, more or less) conditions.
That is, if substance A begins to make a crystal, and substance C fits badly in A's crystal, then the molecules of C will easily leave the fresh outer surface of A's crystal and remain in the solution, while the molecules of A stick better and contribute to the growth of the crystal.
Though, it depends of how compatible A and C molecules are. If you solidify an aluminium melt that contains silicon, since Si is soluble in Al, you get an Al-Si alloy. Even better, some simple ratios of Al to Si atom numbers make nicely ordered crystals, so you really get such intermetallic grains in the solid, with joints in between that take up the Al to Si proportion that doesn't fit the grains.
Slow and regular crystal growth improves the selectivity by making more perfect crystals. Impurities find niches where defects are. Also, a slower growth gives the worse fitting C more time to dissolve again. Silicon single-crystals used as semiconductors take months to grow (see Czochralsky).
Controlled growth improves the process. Ice cubes solidify first at the outer faces, leaving no route for the impurities to escape, so gas bubbles and salts make visible concentrations at the middle, making the whole cube no purer. Oriented crystallization avoids that (see Bridgman).