We've had a few questions similar to this one that deal with assuming that you have a minyan, when in fact you don't.
Shuls are, generally, open to anyone. We usually assume that when a stranger enters the shul, and you have nine people, you are thrilled to see that tenth male coming in so you have your minyan. I'm unaware that any rabbai, gabbai, or anyone else in the shul even thinks to ask the stranger: "Are you Jewish?" (Maybe Chabad does this? They ask me on the subway if I'm Jewish — as if my Mets cap doesn't answer the question!)
In light of the possibility that the tenth man, may, in fact, not be Jewish, is there a halachic requirement to inquire? If so, what would be a good way to approach such a stranger without offending him?
Note: If it were a weekday Shacharit, I guess if you saw that he wasn't wearing a tallit or tefillin, that might be a hint, but not necessarily. One Jewish man in my shul attends daily minyan but never wears tefillin.