If you heat something in a closed vessel, the pressure will change with the temperature. When we determine the boiling point we want to know exactly what the pressure is, so we do it in an open vessel so that the pressure is constant and equal to the atmospheric pressure during experiment.
Boiling point definition doesn't ask for open or closed vessel, it requires a known and constant pressure (doesn't matter how you achieve this, although using an open vessel and atmospheric pressure is the simplest approach).