Dayan Raskin analyzes this custom here. Please read the source for a lot more details than I'm going into here.
Why we turn for Boi Be'Shalom:
Pri Megadim (262:S"K3): This is instead of going out of the synagogue, as brought by the Magen Avraham
Mishna Berurah (262:S"K10): We turn to face the door as if we were receiving/welcoming a great person
Ta'amei Minhagim (p. 127): We do this to comfort the mourners, who sit in the west.
Arizal (Pri Eitz Chaim Chapter 8, Shaar Hakavanot Nagid U'Mitzvah): We go out to the field, then face west, since the sun sets there.
Arizal: Some understand the reason the Arizal says to face west is because, as the Talmud (Bava Batra 25B) and the Zohar (III:119B) tell us, G-d's presence is in the west.
Dayan Raskin points out that these opinions would disagree about which way to face in cases where the door wasn't in the back of the synagogue, or if the congregation is not praying facing east. He then points out the various Poskim who discuss this issue.
How we bow:
Siddur Ya'avitz (in the name of his father, the Chacham Tzvi): Bow first to your left (which is the right of the Shechina), then to your right, the bow in front of you (to the east) and say "Boi Chalah Shabbat Malkata" quietly.
Sha'ar Hakavanot (Arizal): First bow to your right, then your left, and the third "Boi Callah bow to the middle. He does not explain if he means middle facing east or middle facing west.
The Chabad Custom is as follows (taken from Sefer Haminhagim, The Book of Chabad-Lubavitch Customs) (emphasis mine):
When turning around to face west (Pri Megadim I:1, 262; Shaar HaKollel 17:7, citing Shaar HaKavanos.) while reading the paragraph beginning boi beshalom (p. 132), one begins the circuit by turning to one's left (Siddur Yaavetz; see also Shaar HaKollel), and completes it after having said bo'i chalah the second time.
It is our custom to bow to the right while saying bo'i chalah the first time, to the left while saying it the second time, and forward, having now returned to face east, while saying it the third time. (This was the custom of the Rebbe Rashab and the Rebbe Rayatz.
The third time, bo'I chalah is said in an undertone, and it is our custom to read the next two words likewise. (See note 35 and 36 here)