I wouldn't that that an abstract algebra class would be unhelpful, since you do deal with complex functions in quantum. I do agree that there are classes to focus on first.
That said, linear algebra is very helpful for understanding and visualizing, atleast at some level, the eigenfunctions problems of quantum. Calc III will give you the tools you'll need for the differential approach to quantum. Diffies will only enhance your mathematical toolset, and a course in PDEs is even better if your school offers it.
Personally I'd love to take a functional analysis course.
More math is never a bad thing in my experience, so if there's a field of math you really enjoy I'd suggest taking courses in it. Math is pretty interrelated, and it's been my experiences that subjects covered in most intro course can be applicable to a wide range of subjects. I know I took a proof based geometry course before going back to school, and it really gave me a whole knew perspective on math.