I know that Abraham had sex with his slave who acted as a surrogate for Sarah and I understand that to have been standard practice (even though God did not intend for his promise to be fulfilled through a slave) but what about simply using the slave woman for sexual favors? Was that within the rights of a Jewish slave owner? I'm not particularly concerned with ANE norms in general (though I suppose that would be interesting and useful) but rather what the Torah provided for or forbade.
The master of a female Hebrew slave has the option to marry her either to himself or to his son (with her consent). If he (or his son) does marry her, it is not his right, but in fact his obligation to have relations with her, since a husband is obligated to have relations with his wife (Exodus 21:10; Maimonides, Laws of Slaves, 4:7-8).
A Jewish slave owner is permitted neither to marry nor to have relations with a non-Jewish slave (Targum to Deuteronomy 23:18; Maimonides, Laws of Forbidden Relations, 12:11).
However, since Abraham lived before this law was given, it would be anachronistic to apply these rules to his relationship with Hagar.
I do not answer the clear prohibition of having relations with a Gentile servant - it is answered in the previous answer. I just explain Avraham's behavior the question is based on:
According to our Sages view in the Gemmorah, Abraham had dual qualities as a Gentile and a Jew simultaneously. We use it interchangeably to justify his behavior. So it is completely Halachic to say that he was a gentile and was allowed to reproduce through his servant.
Another excuse we use is that it happened before the Jewish nation was commanded on those laws. So the forefathers kept the Torah "optionally" and could deviate if needed.
Another way to justify Avraham is to claim that Hagar was converted prior to her "marriage" to Avraham - why not?. As we don't see any explicit mentioning of conversion for all of the forefathers, we assume they did it privately.
Interestingly, as the commentators notice that Keturah was Hagar, Avraham took her back after Sarah's death. So it wasn't about sexual flavors at all.