According to a Midrash here, he in fact passed her by, at which point she prayed to God that he should in fact sin with her. As summarized in the Da'as Zekeinim here (alhatorah.org translation):
ויחשבה לזונה – וכי מה איכפת ליה אלא בקש לילך לו נשאה עיניה למרום ואמרה רבון העולמים וכי לא אני זוכה להוציא חכם מגופו של צדיק זה מיד שלח לו הקב״ה מיכאל והחזירו כתיב הכא ויט אליה וכתיב ותט לפני מה התם על ידי מלאך הכא נמי על ידי מלאך.
ויחשביה לזונה, "he considered her as a harlot." Why should Yehudah have cared? He turned around to resume on his way to the shearing. However, Tamar raised her face to heaven to pray and ask G–d why she should not be able to give birth to a worthwhile and intelligent son from the semen of such a righteous man as Yehudah. Upon hearing her prayer, G–d immediately dispatched the angel Michael who made Yehudah have second thoughts and turn around. The Torah here wrote: ויט אליה, "he turned to her," and we find the same expression in Numbers 22,33 where it is written of Bileam's she-ass: ותט לפני, "she turned around before me;" (the angel speaking to Bileam) in both instances the angel Michael was the subject. (subject discussed in B'reshit Rabbah 85,8)