An open system is one in which both matter and energy exchange can occur across the boundary. A closed system is one in which only energy exchange can occur. In an isolated system, neither occurs. Entropy cannot decrease in an isolated system ever, but in an open or closed system, anything can happen. Any entropy loss from the system just has to be (at least) compensated by gain in entropy of the surroundings.