This is a worry that plagues all of us who move between the genders! As SallyChoasAura advised above, use wipes specially designed to remove makeup and the best advice I can give is to try a small amount on the back of your hand before you start. If it removes quickly and easily with a wipe, you're in business, if not then avoid unless you want to get caught!
As an aside, I understand that some lipsticks can be quite hard to remove so it's definitely worth checking these before applying to your lips. Waterproof mascara is another thing to be careful with.