- More clearance for larger tires, with 40-45mm being common. Some will also accept 650b (27.5) wheels with even larger rubber
- Typically a lower bottom bracket, more relaxed head angle and longer wheelbase, for increased stability
- Lower gearing is common, too
- They often have compliance features, if not some form of small travel suspension
- You'll frequently find extra water bottle mounts, fender mounts, rack mounts and top tube bag mounts on the more "adventure riding" oriented bikes
From a performance standpoint, they're typically lighter and quicker handling than an MTB, but more stable and comfortable than a road or CX bike. They can be true "quiver killers", one bike that can do almost everything, albeit with a change of wheels or at least tires. If someone new to cycling asked me what kind of bike to get and they were interested in a variety of kinds of riding, I'd steer them toward a gravel bike.
All that said, both of my "gravel" bikes have been CX bikes with fatter (~40mm) rubber and lower gearing for the steep stuff. I ride more on trails and fire/logging roads than on actual gravel or dirt roads meant for vehicle traffic. As long as I avoid the really bony stuff, the bike is great. It's much lighter and more responsive than a full-suspension MTB and a bit lighter and still more responsive than a comparably-priced hardtail.
Overall, I just find the gravel bike to be more fun to ride than my MTBs for most non-pavement riding. I still use an MTB or fat bike for snow & ice riding, and bony trails that the gravel bike doesn't handle well, but that's about it.