We've had a few questions similar to this one that deal with assuming that you have a minyan, when in fact you don't.
Shuls are, generally, open to anyone. We usually assume that when a stranger enters the shul, and you have 9 people, you are thrilled to see that 10th male coming in so you have your minyan. I'm unaware that any rabbai, gabbai, or anyone else in the shul even thinks to ask the stranger, "Are you Jewish?" (Maybe Chaba"d does this? They ask me on the subway if I'm Jewish - as if my Mets cap doesn't answer the question!)
In light of the possibility that the 10th person, the stranger, may, in fact, NOT be Jewish, is there a halachic requirement to inquire? If so, what would be a good way to approach such a stranger without offending him?
Note: - If it were a weekday Shacharit, I guess if you saw that he wasn't wearing a tallit or tefillin, that might be a hint, but not necessarily. A Jewish man in my shul attends daily minyan but never wears tefillin.