A bond is a better organisation of electrons from two atoms. The formation of a bond releases energy, breaking the bond absorbs some.
Nearly always, reactants and products are molecules, with bonds already formed. As compared with individual atoms, the compounds have already lost much energy. Reactions only reorganize the atoms into different molecules; the heat released or absorbed by the reaction is generally less than what the formation of any of the compounds from the atoms would release. That's why lone atoms are scarce on Earth, except for noble gases.
In an exothermic reaction, the products' bonds are stronger (or rather, they release more energy) than the reactants' ones - endothermic is the opposite.
One could add subtleties, for instance interactions between the molecules, heat stored in the vibrations... Often, chemical bonds are stronger than those, and suffice to determine exo vs endo-thermic.
Sidenote: the release of energy is not the absolute criteria to determine whether a reaction goes in one direction or the other. Many endothermic reactions happen spontaneously.