Well, think of the other way, with water attacking the allylic bromide via an SN2 or SN2' attack mechanism. The immediate product would be a protonated allylic alcohol. The pKa's for a protonated alcohol and HBr are -2 and -7, respectively. So in that case, the immediate product at least makes sense in terms of what is protonated and what is not.
You can draw both mechanisms, and maybe both occur to some extent, but the carbanion mechanism would have a titanic activation energy because you would have such a poor equilibrium with a pKa difference of ~42 units in the wrong direction.