I do a lot of off pavement riding with a wide variety of bikes, including road bikes with typical 700x25 tires.
It is important to understand that even though there are many types of pavement, there are many, many more types of surfaces you'll encounter when you go off pavement. Things get more complicated when you leave pavement.
In general, though, it's mostly about surface hardness. Road bikes do just fine on firm, hard dirt without a lot of ruts and without too many obstacles such as roots, rocks and such. Have ridden my road bikes on many an unpaved bike path with firm, very fine gravel, too. If you have such trails in your area, well worth exploring. Have even managed typical gravel roads with larger sized gravel as long as the gravel wasn't too deep and soft or the road badly rutted.
When surfaces get soft/mushy/loose or deeply rutted or less than smooth, things get tricky for a road bike with typical 700x25 tires. Still sometimes (key word, sometimes) doable for an experienced rider, but sometimes even an experienced rider will have to throw in the towel and walk in order to be safe. (By the way, loose sand and gravel and cracks or ruts are also dangerous on pavement, so keep that in mind when riding a road bike.)
It won't take you long to know you have the wrong bike when you try some offload surfaces with your road bike. You'll loose traction, fishtail and overall find the bike very hard to control and ride, safely. That's when you need a different kind of bike.
