if you're riding in clothes not intentionally made for riding, that's a risk you'll have to accept. Some non-bicycle stuff will survive, some won't. It's all guesswork and gamble. All that can be said for certain is that a maker of generic clothes WON'T do much abrasion testing of the fabrics that are used.
If you ride far enough for wear to matter, then you're ALSO riding far enough for bike shorts to make sense for other reasons.