A couple of years ago, I got a pair of Zipp 404's in tubie on ebay for a (relative) pittance - $700. They are sweet. The ride is velvet smooth, though that may be as much the tubie tires as the carbon rims.
Are they faster? On the downhills, definitely. Again, that could be the 58mm rim more than just being made from CF, though making a 58mm rim from anything other than CF would weigh a ton. On the flats, not much if any difference. Wow, do they catch the wind, though. Crosswinds are definitely noticable, though never to a dangerous degree.
Resistance to damage? Given the cost of a replacement rim, I'm paranoid of road hazards when riding the Zipps. I did put them to the test once. Was turning from one road onto another, where some cretinous road crew had cut a trench in the pavement, about an inch deep and three inches wide, with sharp edges. Didn't even have time to bunny hop, just enough to straighten up from the turn. I can still hear that whack! when I hit that trench, thinking I've just lunched my precious wheels. They survived the ordeal with no ill effects, but I wouldn't want to try that twice. So they're not as fragile as glass. For tough rides, I have a set of stout, but brutally stiff, Rolfs.
Fact is, even an aluminum rim should be replaced if it has taken a hit hard enough to deform the rim. The AL becomes crystallized where the dent is, and is subject to later failure. What is your butt (and your skin, and your structural integrity in general) worth? How good is your health coverage? A wheel malfunction is serious business.
I love the Zipps. The combination of smooth ride and fast downhill performance make them my weekend hotrods. I don't race (anymore), but I do love a fast ride on the 404's. They just feel so sweet.