For compounds that are closely related, leaving group ability correlates better with basicity, not with electronegativity. For example, in terms of leaving group ability Cl- < Br- < I-. This is anticorrelated with electronegativity. However I- is the weakest base of the three, which is another way of saying that HI is the strongest acid of the three.
With respect to acetyl chloride versus acid acetic anhydride reacting with an amine, you can still use basicity to understand reactivity. Which is the stronger base, Cl- or acetate ion? The stronger base is usually the poorer leaving group (there are some exceptions when you compare compounds that are very different in structure). I am not 100% certain that this is what you questions was about, so please let me know if it needs to be clarifed.