Yep, those are the two most likely culprits. Rest between workouts including rest days in your weekly schedule are real important, but are you also making sure you replenish the glycogen you burn during your harder efforts? Failing to replace your spent muscle glycogen is probably the leading cause of cumulative fatigue for athletes with a reasonable weekly schedule.
Make sure you take advantage of the critical half hour when you first return from a workout. It's the best time to get carbs on board and a short window during which those carbs will quickly be converted to glycogen and stored for future workouts. Failure to eat enough carbs during that initial window really delays the process of synthesizing and storing glycogen. Here's some links to critical half hour refueling and some guidelines on carb to protein amounts based on bodyweight.
http://www.carbboom.com/education/recovery.php
http://sportsmedicine.about.com/cs/nutrition/a/aa081403.htm
It's also possible you just bit off a bit too much with your harder than usual session and need an extra day or two of easier work for your body to recover. When I feel like that I just back off my scheduled workouts a bit and typically get back on track and feel a lot stronger in a couple of days. The worst thing to do if you're just not recovering is to try to push through it with additional high end workouts. That can easily lead to an overtraining hole that's hard to climb out of. I don't like to rest too much if I'm a bit tired, but it's easy to drop my scheduled workouts down an intensity level or two until I'm feeling strong. Usually that means a couple of days of high tempo or SST riding instead of threshold or VO2 max sessions.
Good luck,
Dave