Yes, DU is less radioactive than natural uranium. The U-235 is removed, and, more importantly, so is the U-234.
The latter is only a tiny fraction of the U present, but it has the same radioactivity (in decays/sec) as the U-238 which it is in equilibrium with.
The hazard depends very much on whether you ingest it. If you have a piece of solid DU sitting on the floor in a room with you, the hazard is negligible. Part of the reason for this is that U is mainly an alpha emitter, and alphas have almost no penetration ability, they will not get through your skin or even a foot of air, for example. If you inhale very fine dust, (such as that produced by a DU armor-piercing round on impact), this is another matter entirely, as you get it in your lungs where it sticks, and the alphas are easily able to get to your cells from there. U is also chemically toxic.