Rear End Slip

I have just built my first bike. New frame, saddle, shorts and seat post. I have noticed that the saddle seems to slightly move against my rear end whilst riding. I cannot seem to make it move by standing next to it and giving it a wiggle. All set screws are at the right torque settings and the post doesn't move at all within the frame. The rails are also secure within the saddle. The only thing left is the seat post. It is carbon and just wondering if they flex enough to feel any movement ? It's the first one I've used on a road bike so I have nothing to compare it to. Or it more likely to be my dacks ( shorts ). Do different brands slip between your R's and saddle ?