The Rambam writes (Hilchos Isurei Biah 21:1):
כל הבא על ערווה מן העריות דרך אברים, או שחיבק ונישק דרך תאווה ונהנה בקירוב בשר--הרי זה לוקה מן התורה
One who comes upon one of the Arayos via their limbs, or hugs or kisses in a lustful manner and benefits from the touching of skin receives malkus according to the Torah.
The implication is that the category of Shomer Negiah is only applicable with regards to lustful touching. Now I am aware that there is a discussion of cases where lustful pleasure is highly unlikely, e.g. shaking hands in a business setting. However, it seems to be the common practice that men and women will casually touch members of the same sex, since it is assumed that they cannot be attracted to each other at all. What about the case where one or both of the parties has a homosexual, bisexual, or asexual orientation? How will this change the categories of people that may be touched?
For clarity, here are two examples to illustrate this question:
May an asexual man touch a woman? (Normally we assume that the man could be attracted to the woman, but in this case we know that he is not.)
May a bisexual man touch a man? (Normally we assume that the man could not be attracted to the other man, but in this case we know that he could be, potentially.)
I am also interested hearing about how yichud might or might not apply in such cases, if it is relevant.