If you never had this problem before then it's not the frame unless you had a recent crash, otherwise it's something mechanical and I agree to look at the stem and headset; but first make sure your front skewer is properly tightened, the skewer lever should just start to feel tension when it is 90 degrees out from the locked position, then retest the bike, if the problem is still there then the headset is probably loose. You can check most headsets simply while at a stop apply the front brake and rock the bike back and forth, if you feel something knocking or it feels loose then the headset is out of adjustment.
It could also be the front hub bearing is out of adjustment or worn out, but I doubt this, but if the above stuff doesn't fix it then you need to look here.
Here is a video on how to adjust the headset, this may be a Cane Creek adjustment but it works for all others; see: http://www.youtube.com/watch?v=W2RHkdRePa8