If your compound is quite low in polarity you can wash it out, as mentioned already.
I usually dissolve in EtOAc and wash with 1:1 brine/water a few times. This works very well for DMF - I haven't tried it for DMSO. Note that if you extract with an organic several times you will probably just back-extract all the DMSO, so I'd do aqueous washes as opposed to organic extractions. I'm basing this on my experience with DMF, so it may not transfer exactly.