This has happened to me two or three times over the past year. Restarting was always enough to solve the problem. Unfortunately I have no idea at all what was causing it. It could have been a .Net or WSR issue. When something happens so rarely I try not to worry about it since it probably has nothing to do with my code, or even if it does it is nearly impossible to track down. Because VC also connects to so many other things, it could theoretically be caused by one of the other programs it is attached to, although this seems like a long shot. I have definitely seen where one program crashing can bring down other programs that it is connected to. Example, MediaMonkey crashes and causes VC to crash.
If it is happening to you on a regular basis then probably it either has something to do with your system (hardware or software) or your configuration. I don't see how I can help if it is your system, but I can try to run your configuration to see if I can recreate the error.
Other than that I don't know what to suggest.
Edit / PS:
Normally I see 0%-2% CPU usage for VC with a max of 5% while issuing commands.
Rebuilding groups can use up to 100% for a brief period.