Sometimes aluminium salts plus alkali are used to precipitate aluminium hydroxide from water, as this bulky precipitate carries down with it many impurities and potentially toxic substances.
Aluminium compounds are only water soluble, and therefore likely to be absorbed by the body, if the pH of the ingested water is very low (as [Al(H2O)6]3+) or very high (as [Al(OH)4(H2O)2]-). Water supplied to the water-mains, for drinking or washing, is at near-neutral pH so soluble and therefore absorbable aluminium is absent from it.
Aluminium is one of the commonest elements on Earth (7% or so of Earth's crust). Life has had 4 billion years to learn to handle it. Therefore life is likely to have evolved resistance to Al toxicity in all that time. That which did not would not have survived.
Those elements which are most toxic to life are mostly very rare ones like Thallium, Arsenic, Lead and Mercury which living matter has, before human intervention, not had to deal with in any quantity.