Think about what the phase diagram is actually telling you. What happens physically when you increase pressure on something? The molecules get closer together and the substance becomes more dense. So if you are following a phase diagram along increasing pressure and your substance changes form, the form at the higher pressure will be denser than the form at the lower pressure. Otherwise, it would not be favored by an increase in pressure.