If a Kohen's wife is G-d forbid raped and he witnessed, there is no heter (permission) to stay married with her as she is a zonah (Leviticus 21:7). However, after he divorces her, can he take her back, not as a wife, but as a pilegesh (informal marriage without kiddushin or Ketuba)? It only says "lo Yikakh" - take her in marriage, here he is not formally married to her. Same question would be if he falls in love with a lady that is divorced whom he takes as a Pilegesh.
Whether there is Malkus (39 lashes) for a Zonah/Gerusha/chalala having relations with a Cohen outside marriage is a Machlokes (dispute) between the Rambam and Raavad based on Gemora Kidushin 78a, Hilchos Issurei Bia 17,2. The Rambam holds there is no Malkus as the prohibition is only on relations when Marrying the Zonah/Gerusha/Chalala:
כל כהן שנשא אחת מהשלש נשים אלו בין גדול בין הדיוט ובעל לוקה ואם בא עליה דרך זנות אינו לוקה משום זונה או גרושה או חללה שנאמר לא יקחו עד שיקח ויבעול:
Where as the Raavad learns from a Gezeira Shava from Kohein Gadol who is chayav Malkus to have relations outside marriage with an Almana (the Rambam agrees that Kohein Gadol gets lashes for casual relations with Almana), so too a Kohein Hediot is forbidden with lashes to have relations outside marriage with a Gerusha/chalala/zonah:
השגות הראבד_ ובעל לוקה וכו': כתב הראב"ד ז"ל אין השכל מודה בזה שהרי למדו כהן הדיוט מכהן גדול לחלול הזרע בג"ש זונה זונה הלכך כי הדדי נינהו באסורייהו וטעות הוא בידו והטעהו מה שראה בגמרא כהן גדול באלמנה לוקה שתים אחד משום לא יקח ואחד משום לא יחלל והוא סבר דוקא כהן גדול ולא היא אלא משום דלא מיתני ליה בכהן הדיוט חלול דאשה בשלא גמר ביאתו קמיירי דליכא חילול זרע עכ"ל
The Maggid Mishna explains form the gemora Kidushin 78a that both the Rambam and Raavad agree that the child from such a relationship outside marriage of a Kohen and a Zonah, is a Challal and is forbidden to Kehuna because of a Gezeira Shava so this relationship is still forbidden min haTorah according to everyone, regardless of whether being punished by Malkus or not, as one may not make Zera passul:
ילפינן שהבנים חללים כמ"ש שם נאמר בהדיוט זונה כו' מה להלן זרעו חולין אף כאן זרעו חולין
1 Kohen wife rapped
If Kohen's wife was rapped, she is halachically prohibited for her husband because of tumea. Regarding Zona, machloket amorayim in Yevamot 56b
2 Regarding the prohibition of simple Kohen or Kohen Gadol with women who are prohibited for marriage with Kohanim.
. Prohibitions of intercourse regarding a Kohen:
-- whether because of a woman who will be profanated, id est prohibited from kehuna by this act,
-- or because potential or actual offspring.
Literally, Torah marriage prohibition only, not intercourse prohibition in two separate verses, one for Kohen Hediot and one for Kohen Gadol.
. For Kohen Gadol the prohibition of marriage is followed by a statement regarding profanation of his seed.
. For Kohen Hediot, there is no similar statement.
. Profanation through "marriage" by Kohen Gadol has two parts according to Gemara Kiddushin 77-78.
. Firstly, to profane a woman (who was permitted for Kohen) by intercourse, making her pesula for kehuna. There is one case only: a widow. She is permitted to marry a simple Kohen but prohibited for the Kohen Gadol. If Kohen Gadol marry her, he makes her pesula even for Kohen Hediot.
. The second part, profanation by engendering children who will be profanates (chalalim). We will see below that this second part is not LaHalacha according to Rambam as long as no child not born yet.
. This problem of profanity is greater for sons because of its persistence through generations (children of sons are also profanates).
. For a girls, if she marry a non Kohen man, her daughters can marry a Kohen, and sons have Kasher offspring.
This difference between sons and daughters is learned by a gzera shava between Kohen Gadol and Kohen Hediot. (1)
3 Gzera shava between Kohen Hediot and Kohen Gadol
There are two opinions in rishonim regarding the scope of this gezera shava (GS).
. Raavad and Ramban think that GS makes an equivalency between marital prohibition of Kohen Gadol and Kohen Hediot. So, prohibition of profanation through sexual relationships (not linked to marriage) exists for Kohen Hediot as for Kohen Gadol (but the scope of profanation is littler for the Kohen Hediot because there is no case of being prohibited for a woman allowed to other Kohanim and making her prohibited to them by intercourse).
. On the other hand, Rambam and Maggid Mishne think that the spectrum of the gezera shava is narrower. It's strictly what the Gemara says in daf 77a. Profanity is transmitted through generations is by males children only. But for Kohen hediot we learn by the GS that his offspring would be profanated if the mother is one of the prohibited for him. The gezera shava doesn't addresses Cohen marriage prohibition.
4 Profanation as profanation of Potential/Actual offspring
. Rambam rules that even Kohen Gadol has a prohibition to profanate woman but not to profanate his potential children (if no children is born, there is not Avera). This is the opinion of Rava in Gemara according to his lecture. Rambam reads that it is a machloket Rav Yehuda and Rava. He understands that conclusion of Gemara follows Rava and the only profanation problem is to profanate a widow, not to "potentially" profanate children. However, if a child actually borns, the intercourse is retroactively a profanation of offspring that makes sense even if the woman was already profanated before the intercourse(according to understanding of Magid Mishne, not written explicitely in Rambam).
. Raavad and Ramban rule that the prohibition of profanation of potential children exists for Kohen Gadol and for Kohen Hediot (no Machloket between Rav Yehuda and Rava) . So, intercourse with a woman pesula lekehuna even without Kiddushin for a Kohen is chayav Malkut.
. Maggid Mishne has an intermediary opinion. He extends the prohibition to Kohen Gadol only (as soon above in the explanation of gezera shava), but he adds all cases of women who can have profanated children with Kohen Gadol, even with a divorced woman who was already statuted as profanated. And the intercourse needs not to actually engender a child (he rules as Ramban and Raavad that Rav Yehuda doesn't address Kohen Gadol specifically ).
This presentation results from lecture of Maggid Mishne Isure Biah, 17.2, to understand it, you need to know the Shakla Vetarya of Gemara Kiddushin from 77a to 88a.
So, if a Kohen Hediot (nowadays, there is no Kohen Gadol) needs to divorce his wife because she was rapped, she is prohibited for him by marriage prohibition (she is a divorced woman). But Intercourse after divorce, according to Rambam and maggid Mishne is not punished by Malkut if no children is born from the intercourse. However, according to Ramban and Raavad, by intercourse he transgresses the prohibition of profanation and is punished by Malkut.
I have noticed that in Gemara Tmurah 29b, Abaye and Rava discusses a non marital relationships between Kohen (without mentioning Gadol or Hediot) with a prohibited woman and speak about Malkut. This is easily understood according to Ramban and Raavad. But according to Rambam, there is a machloket hasugiot what Abaye and Rava think. This is a confusing topic, hard to clear.
(1). Gemara (Kiddushin 77a) learns that profanation of children is transmitted through generations by males only. Females don't transmit their profanity. We learn this from a gezera shava "Amav" "Amav". Amav is written in simple Cohen for the prohibition of uncleanness, that apply for males only. So the profanityl transmitted by children is by males children only.