For me, the jump from Sora to Ultegra was a bit underwhelming, but in my mind it was worth it.
A higher end group is mostly lighter and can do some nifty things like down-shift multiple cogs in a single click from the shifters (I'm not sure if your Sora can do this, mine couldn't - it was pretty old). Unfortunately, the new Ultegra 6700 only downshifts 2 cogs at a time, something you might not like. I had an older Ultegra group for racing and it could downshift 3 cogs at a time, and I really liked the responsiveness.
Some people say that Sora is just as good as Ultegra if maintained correctly. I don't think this is true since you have to adjust Sora a lot more often than a higher group, and the parts on Sora are kind of flimsy in my opinion. For recreational riding, however, it's alright - however, for really, really, long distances, like long tours (thousands of miles), you will need something that's more durable and reliable.
The higher groupsets themselves don't make your bike that much faster, but it will be a little bit more responsive.
What makes a bike faster? I'm not exactly sure, but when I upgraded to a high-end racing bike I went much faster. It's probably a combination of the frameset and wheels. The frame was lighter, and the wheels spun faster as well. It also cornered and handled a lot better, which allows you to maintain speed while cornering and descending - two very important skills in racing. Lightness is a plus too. You know that feeling you get when you go up a hill and your bike slows down a lot? If you get a lighter bike, hard climbs won't be a piece of cake, but that feeling of heaviness will come later and you will be able to maintain a faster speed while going uphill.
I think at 35km/h you will notice some difference from a better bike, but probably not really from just a better groupset. In races where the "little" things matter - like cornering, accelerating, and responsiveness, you will notice a bigger difference with a high-end bike.I'm guessing your bike came with Sora - so the rest of the bike probably isn't too fancy either. In my experience, it's best to wait to buy a new bike if you're thinking of upgrading from Sora because in the long run it's better than buying little pieces one by one - you won't regret it.