First, I'd not get rid of the 404's unless you're willing to take a bit of a bath on selling price. Second, as suggested by oldbobcat you need to think about what you really need in a set of wheels. Once you've got an idea of what you want, I'd talk to a wheel builder about getting a custom built set of wheels. Custom wheels tend to be cheaper than boutique wheels, except for the boutique wheels near the bottom of any given manufacturer's line. Other benefits of custom wheels are getting a wheel set configured just for you (rims, spokes, nipples, hubs) and likely getting better service at the time of build and later if needed.
For the cost of Ksyrium SL's, you can get a lighter, more durable wheel set that also comes with a much better rear hub. The Mavic rear hub has a delrin bushing in place of a bearing in one location, which IMHO is an unsatisfactory solution. At least one LBS (a big, popular one) here has seen frequent need to replace customers' delrin bushings due to quick wear. I'd also avoid the Rolf wheels. I know some people like them, but the paired spoke idea is just stupid from an engineering point of view. If a spoke breaks, a Rolf wheel is much more likely to become unrideable than a traditionally laced wheel. Likewise, truing paired spoke wheels can try or even annihilate a person's patience.
Back to the Zipps: if you have the older style Zipp 404 clincher--alloy rim--you will definitely take a bath on resale. The new Firecrest 404 has seen to that. If you've got the new Firecrest 404, don't get rid of it unless you just can't stand the wheel set or don't foresee really needing or wanting it. Oldbobcat is right in that at the speeds at which you're pedaling, you won't see a great increase in speed, BUT over a long ride you could see a significant reduction in energy spent. Here's a graph that shows power needed to overcome the wheel drag vs. the speed at which you're moving for an aero rim vs a box rim (from an older Zipp chart, where the box rim was a GL330). This would also be an energy spent over a given time at a given speed graph. You would expect to see similar graphs for any aero rim vs a box rim. You can see that at 15mph you're only starting to see a significant difference in power needed to overcome drag for the two different wheels.
Of course it goes without saying that if you really dig the wheels, keep them. That you're happy with what you have on your bike is just as important as any other reasons given. It's no longer true that CF wheels have to be race day only or special event only wheels. I see a lot of folks riding deep rimmed CF wheels out here in the windy, sandy desert without issue, and the roads out here can be as crappy as any other roads elsewhere in the US. The blowing sand and grit does add an extra wear component to the functional lifespan of a part.