I picked up a Pearl Izumi Zephrr Vest, a nice color of blue.
I like it because it's very lightweight but does a nice job of keeping the wind off my chest. It has a pocket on the back that you can turn the entire vest inside out and it folds up onto itself for storage in my jersey pocket.
I usually care it and a pair of arm warmers. I've noticed my arms don't get as cold as my chest, so I usually don't wear the arm warmers unless it's much cooler.
Like others have said, I also will wear layers if it's that cold, I have a couple of long sleeve base layers that will fit into my rear jersey pockets if I get too hot while on the road.
I would suggest a vest and arm warmers over a jacket anyday.
Good luck!