If your spouse is spending time around someone who makes you uncomfortable, I think that you should tell them, and they should honor your request not to spend as much time or be alone with them.
I dont think it should usually be a case of completely not talking to someone, unless it happens to be an ex of some kind with some pretty strong ties. Or someone who is getting in the way of your relationship a lot and causing rifts.