See Ramma in Even HaEzer 154 Siff 1 .
He discusses a man who is Ro'eh Zonos, which may mean he visits prostitutes or just has random affairs, and his wife complains about it. He says witnesses of him cavorting like this (not just young nonjews stating this) is grounds for forcing (either literally or through religious convincing, see Siff 21) him to give a divorce according to some, (Yesh Omrim). [In this context where there is no opposing opinion, Yesh Omrim is a way of introducing an accepted novelle]
Aruch Hashulchan on the subject in Siff 16 explains that someone who acts like this will necessarily detract from his conjugal obligations to his wife which is grounds for divorce. He also adds that this behavior is also grounds for her to say Maus Alay, he is disgusting to me which in this case is again is grounds for divorce, and especially since he may be exposing her to health risks.