Electron affinity is
not "the change in energy that occurs when a neutral atom gains or loses an electron". It is the change in energy when an atom or ion gains an electron. I say "or ion" because you can have first, second etc. electron affinities. For example for oxygen the first EA is the energy of the process
O(g) + e
O
-(g)
and the second EA is the energy change of the process
O
-(g) + e
O
2-(g)
Ionisation energy is the energy change when an atom or ion loses an electron.
The difference between EA and electronegativity is that EA is a well-defined, measurable quantity, while electronegativity is a vaguer concept. EA is a property of free atoms in the gas phase, electronegativity is "the tendency of an atom
in a molecule to attract electron density to itself". That is harder to quantify, and there are several different ways of estimating electronegativity - see e.g. the Wikipedia article on "electronegativity".