Bones do become weaker with age, but there is actually a lot you can do to slow that down and even reverse bone loss in some cases. There is some misunderstanding about calcium supplements out there. It turns out that taking calcium supplements or even eating extra dairy will not cause your bones to retain more calcium after menopause unless you do bone building exercise, which is weight bearing exercises that stress the bone, stimulating the bone to draw in more of the calcium available to strengthen itself. It's somewhat similar to how muscles build strength, they must be exercised. Also, taking a lot of calcium when you are not doing exercises that promote bone building may lead to too much calcium in the blood, and calcium deposits elsewhere in the body, which can be very bad. I would personally stick to food sources of calcium and skip the supplements.
Cycling will not stress your bones much at all, so you'll need to make sure you do weight bearing exercises in addition to it.