The key thing to remember is that the cathode is the electrode where reduction occurs, and the anode is the electrode where oxidation occurs.
When I started working with batteries, I was confused that the cathode was the positive electrode, whereas from the electrochemistry I remembered from school and undergraduate days (long, long ago) the cathode was the negative electrode (where e.g. you deposited Cu from CuSO4 solution). It is necessary to appreciate the difference between a galvanic cell and an electrolytic cell.
In a galvanic cell (battery), a spontaneous chemical reaction generates an electric current. The occurrence of the reduction reaction at the cathode consumes electrons from the electrode, giving it a positive potential, and the oxidation releases electrons to the anode. Electrons flow through the external circuit from the anode to the cathode (and conventional current the other way).
In an electrolytic cell, the passage of an electric current causes a non-spontaneous chemical reaction. At the cathode (the electrode connected to the negative terminal of the battery) reduction occurs when electrons are pushed in at a sufficiently negative potential, and likewise oxidation occurs at the positive anode.
So in one the cathode is positive, in the other it's negative, but in both the cathode is where reduction occurs.
(A rechargeable battery cell behaves as a galvanic cell during discharge and an electrolytic cell - where the "cathode" behaves as an anode, i.e. oxidation occurs there - during recharge.)