See O.C. 132:2 Ba'er Hetev #9. He says that Mahari"l bowed down 3 times when he left his place, and faced the ark while doing that. He repeated this when he was at the entrance of the shul. He then adds that when leaving, one should not have his back to the ark, but should face sideways. One should also face sideways when he leaves the teiva (after being Shat'z or Torah reader.)
I'm inferring from this that one should never have his back to the ark when he leaves, and moreover, he shouldn't even face the ark (other than for bowing), but should face sideways when leaving. So, if the door is either next to the ark or opposite it, one should walk out facing sideways. If the door is to the sides, he can face the doors.
In my shul, the door is to the left when facing the ark. So, exiting the shul, the ark is to my right. I don't think I have a problem, then, according to this. The ark is actually on the south wall, not east. That may be a separate problem.