I think the easiest way to differentiate them is to think about what has to happen first. You have two options.
1) Direct displacement (or elimination) of your leaving group. In this case you form a bond and break a bond all at once. The steps are concerted.
2) Step-wise substitution or elimination. In this case, the leaving group falls of spontaneously forming a carbocation. After that happens, there is either a nucleophilic attack on the cation or an elimination.
Choice 1 is either an SN2 or E2 reaction because it involves two molecules coming together to cause the reaction to proceed. Therefore when you look at the kinetics of the reaction you find that the rate depends on the concentration of both the nucleophile (or base) and the substrate with the leaving group. Overall, the reaction is second order (one from each component).
Choice 2 is either an SN1 or E1 reaction because the rate is determined by a unimolecular process (the spontaneous loss of a leaving group). Once the leaving group falls off, the second step (nuceophilic attack or elimination) occurs very rapidly. When you work out the kinetics for a process like this, you will find that the concentration of the nucleophile or base has essentially no effect on the rate of the overall reaction; only the concentration of the substrate with the leaving group matters. This is due to the fact that the loss of the leaving group to form the carbocation is much, much slower than the attack or elimination. All in all, the rate law only depends on the concentration of one molecule and therefore the overall process is first order.
So, when I approach a problem like this I try to figure out whether or not the leaving group is likely to be displaced by the nucleophile directly (SN2) or a carbocation is likely to form first, followed by nucleophilic attack (SN1).
I hope this helps.