this is mostly fear. How long have you been riding? Usually with time it will subside. You can also be suffering from visual problems and how your brain interprets the information being feed it. It's like not everyone can be car racer or a quarterback because not everyone can see the whole picture like those people can. Roughly 75% of the population suffer from tunnel vision effect when being subject to even simple driving down the road, but it worsens as they go faster, some of the 75% suffer from this tunnel vision starting at far lower speeds than others within the same 75%, and it could be you have this problem, your tunnel vision is happening at a slower speed and when it starts it freaks you out more on a bike then in the safety of your car. A quarterback can see the entire picture without even looking directly at an open receiver and know he's there and in split moment ascertains that the intended receiver isn't open and throws it the guy he never even looked at directly. Most people can't do that because their brain is tunneling in on one person and not all the possible targets on the field, and the quarterback isn't even moving fast like on a bike or in a car.
Personally I think most if not all your problem is fear. Try this the next time you are riding faster than you're comfortable going, in fact you should be doing this all the time (even in your car) not when going faster than normal, stop looking a foot or so at the ground in front of your front wheel, instead look ahead at least 1/8th of mile, if you start gazing downward catch yourself and go back up, and see if this settles the weaving down. If you're on a winding downhill road then always be looking at the furthest distance you can see the road. And while you're looking ahead keep your eyes scanning from left to right so you can see the big picture. As far as cars passing and you pulling over, again it's fear but it could be related or combined with the above stuff I mentioned in the first paragraph, you may think the cars are closer to you then they actually are because of the tunneling effect and it freaks you out.
I think if you practice keeping your eyes looking ahead and scanning you should realize an immediate relief in weaving and hopefully eventually the fear of cars passing beside you will subside.
I hope I made sense.