A derailleur hanger interface can collect debris or show damage, so mating faces, fastener condition and alignment symptoms deserve separate checks. It is easy to just look at the hanger itself, but if there is grit or corrosion between the hanger and the frame, your alignment is never going to be truly stable.
I always look for those separate signs of wear. If the fasteners are chewed up or the mating faces are pitted, you are fighting a losing battle with your indexing. Proper maintenance means cleaning out that gap and verifying the hardware is actually securing the mount flush against the frame.
When you are checking a used bike, do you look for debris and wear specifically at the hanger interface, or do you just rely on an alignment tool?