If you use the same towel several days in a row it can cause this too. Change your towel you dry off with every other day, if not every day.
