Thinking about the material science side of things, carbon would be better with the low amplitude high frequency stuff such as road vibration. The changes in the weave and ever changing fibres/matrix do not allow vibrations to pass as easily.
To damp the low frequency high amplitude (big bumps) you need to flex. Bars and stems that flex a great deal shouldn't be on any bike!
I can see that carbon bars and stems may cut the "buzz" and that can creep into you over long rides without noticing it. However look at what the pros ride....many of them don't even ride the lightest of Al stems, prefering stiffness.
For the last hundred years people have been racing on non carbon bars/stems over rough courses and 200miles torture tests. I'm betting a good fitting bike and fit rider wouldn't need the damping that a few extra hundred $$$ gets you in carbon bars/stems. If you can afford it and want it, then yes there may be some extra enjoyment could be gained due to a little less arm/shoulder fatigue, but unlikely they will make you the slightest bit faster.
Back to the point of this post. If you enjoy the setup you have now why not keep riding it until you do crash and need to replace it? If you never crash you've saved yourself a lot of money replacing some kit you were already happy with!