Neither A nor B melts first - what melts first is a mixture of the eutectic composition. For example, if you take a mixture of composition 80% A + 20% B, which has excess A compared to the eutectic composition, as you heat it (along a vertical line on the graph), when you get to the eutectic melting temperature, a mixture of the eutectic composition melts, until all the B has melted (so you can get no more eutectic mixture). Then you have liquid eutectic and solid A. As the temperature rises, A melts until you cross the solid-liquid equilibrium line , when all the A has melted and you have liquid of the original composition.
What makes the difference is which side of the eutectic composition your starting mixture is - not which component has the higher melting point (as a pure substance).