I live in the Atlanta area
The most notable place is the gaps near Dahlonega, Georgia. There is a popular organized century call Six Gap that you can web search and if you can see the link below or sign up for a free account. This may help you dial into the probably the best area for climbs.
http://www.strava.com/segments/770875
Apart from the gaps there is Burnt Mountain and Fort Mountain that each are a single peak, but a nice loop can be made out of either.
I live on the opposite side of town from the gaps so I have not been over there in a few years. Here are some of my rides that can be downloaded to use as a course in a gps device.
4 Gap (short of the full six gap route) (Hog Pen is the most challenging climb in Georgia if you want to web search)
http://connect.garmin.com/activity/17873056
Armuchee - Fouche Gap (short single climb, but a really nice route scenery wise. It goes through the Pocket, which a lot of cyclist like. It's fun and scenic in that stretch)
http://connect.garmin.com/activity/345461454
Not many store options, but you can work in Amicalola Falls State Park and refill water bottles. They typically let cyclist enter for free to refill and use the rest rooms.
Burnt Mt 72
http://connect.garmin.com/activity/18158866
Burnt Mt 57
http://connect.garmin.com/activity/17785620
Jasper to Fort Mt (I really like this route and the love the Fort Mt climb)
http://connect.garmin.com/activity/70231682
Brewery to Horn (Single short climb, but a nice route. You can park at the Budweiser Plant for many different routes)
http://connect.garmin.com/activity/297381999
Visit my blog and you might find some other helpful information or ask here if you need to know more details about one of the routes I listed.