Chromatography of whatever sort is all about differences in attraction of the compounds that you are trying to separate between two substances of different characteristics.
So think about it in a physical sense. If you are trying to separate compounds based on their polarity, you will have two substances (a stationary phase and a mobile phase) that have different polarities. The compounds that you are eluting (product and impurity) will need to have a polarity somewhere between the polarities of the stationary and mobile phase.
For normal phase chromatography, the stationary phase is more polar than the mobile phase. Typically silica gel is used as the stationary phase. Then you choose your mobile phase based on the characteristics of your compound and impurity. If your compound is more polar than the stationary phase, then it will interact stronger with the stationary phase than any mobile phase you can choose, and it won't elute off the column. If your compound is less polar than the mobile phase, it won't interact as strongly with the stationary phase as the mobile phase does, and it will run off the column with the solvent front. Ideally, you choose a mobile phase that interacts with your compound only about 30% as well as the stationary phase does (giving an Rf of about 0.3) and that gives good separation between the compound and the impurity. If you have a very polar compound, you're going to need a fairly polar mobile phase, but not more polar than the stationary phase. If you have a very non-polar compound, you're going to need an even more non-polar solvent to get any separation at all.
For reverse phase, the stationary phase is usually an 8-carbon to 18-carbon unbranched alkane (C8 or C18 columns) - things don't get much more non-polar than that. Water is usually used as the mobile phase, and you don't get much more polar than that. Polar compounds will move faster down the column because they interact well with the water and not so much with the stationary phase, while non-polar compounds won't move much at all. So if you have a compound that is so non-polar that it takes forever to come off the compound, you need to reduce the polarity of the mobile phase. This is done by adding water soluble organic solvents. Water soluble organic compounds are more polar than water insoluble organic compounds - if you try to use too non-polar a solvent to cut the water, your mobile phase will separate into two layers. Acetone, acetonitrile, methanol, ethanol, and similar compounds can be used. If you want to use a gradient in a reverse phase system, you will start with a more polar solvent (eg 100% water) and slowly add your less polar solvent.
If your compound is so non-polar that even the water soluble organics at 100% won't move them down the column, you can remove the water altogether and just use more and less polar organic solvents, but at that point you are usually better off switching to a normal phase system, where your compound will tend to interact better with the mobile phase than the stationary phase.