It depends on what exact base you are using. Some bases are nonvolatile salts such as NaOH (lye) or KOH. These will behave as you have said, evaporation of the solution leaves solid crystals of the base. However, the water vapor will not be basic.
Other bases, such as ammonia, however, are volatile. Evaporation of these bases will produce a basic vapor (although sometimes you may be able to separate the gaseous base from the water vapor), but it will leave little or no residue once it is boiled off.