Lot's of good advice above. Some more thoughts:
- As Alienator suggests, don't ignore your feet. That's the first thing to get really cold as you don't move your toes much while cycling so get some decent booties to go over your cycling shoes. If you're going to ride in really cold weather they should be neoprene or other insulated booties, if you're riding in just cool but not frigid weather then a rain barrier style bootie can help keep your feet warm enough. If it's real cold consider picking up a pair of slightly oversized cycling shoes that you can wear with thicker wool socks without cutting off the blood circulation to your feet and toes.
- Wool layers rock. Consider a long sleeved wool cycling jersey as a very versatile layer for cool to very cold conditions.
- Leg warmers may be enough if it's just cool out (as in 40 to 45 degrees or higher) but for truly cold conditions go with full tights over or preferably under your cycling shorts. In addition to keeping your lower legs warm, an extra layer in the crotch can really help you from getting chilled and numb there which leg warmers can't do.
- Good gloves are essential, if it gets real cold or wet and cold then a pair of thin surgical gloves under your long fingered gloves can really help in terms of keeping your fingers warm. They'll still get soaked with sweat in that vapor barrier environment but they'll be wet and warm, not wet and frozen. Definitely dry your hands out well when you get home but that's not really a problem while cycling as it might be during extended winter outings like winter mountaineering.
- The full face mask can be great on really cold rides but get a thin helmet liner hat for most cold days. That alone can make a huge difference in terms of retaining body heat. A thin nordic ski beanie is a must have item on cooler days or warm cycling helmet liners that are snug around the ears are pretty nice as well.
It's not too hard to dress to ride year round, just be extra careful or skip your rides if there's ice, especially thin black water ice on the roads. That can end the fun in a hurry. But riding outside in the cold can be fine as long as the roads are dry and relatively safe.
-Dave