I like to think that molecules of the mobile phase compete against the analyte for binding to the stationary phase. The more polar the mobile phase, the less well it binds to the very non polar stationary phase. This allows the analyte molecules to stay bound for long periods of time and to have a long retention time. If the mobile phase is less polar then it competes more effectively versus the analyte for binding to the stationary phase, and the analyte molecules are more likely to be displaced from the stationary phase.