Because to travel the same distance, the wheel has to turn more times, this leads to higher resistance in the hubs (I think). That is the theory.
Some suggest that 650's are okay if the rider's height is up to 173-178cm (5 foot 8 inches- 5' 10") only and that 700's are good for those 168 cm (5' 6") and above. The overlap is for overall bike fit and rider flexibility.