23 mms have less rolling resistance than a 21 or a 19, so would be 'faster.' The 23s would also probably be a little more comfortable, due to the lower rolling resistance, as well. Wider tires also corner better due to a larger contact patch.
The 21 or 19 would provide an aero advantage on the front wheel as compared to a 23 mm. That aero advantage would be outweighed by the higher rolling resistance in a peleton, but the aero would probably outweigh the rolling resistance in a TT. Potentially a gram or so of weight savings (19 mm) as well.
Of course, we're talking about tenths of a second here - at most. in other words, you wouldn't know the difference.
I've ridden a 19mm front with a 23 mm rear in all types of races and haven't had any problems. My next front will probably be a 23 since I ride mostly road/stage/crits.