OK, I always used to argue this using electronegativity with a good success rate.
So, start with a simplified MO diagram of the levels involved:
Metal pi*
^
|
|
|
Ligand pi
That thing between the levels is meant to be a vertical arrow indicating the LMCT.
Now, as the electronegativity of the ligand drops, what happens to the energy of the ligand pi orbitals?
So, does the energy of the transition increase or decrease with electronegativity of the ligand?