Shulchan Aruch (Orach Chaim 53:18) makes it clear that this is only because of the concern that he is an apikores.
The Mishnah Brurah there writes that if he immediately gives an explanation for why he did it, he can be the shliach tzibur.
Also, the Tiferes Yisrael (ch. 4 note 52) writes that it is only for that prayer that he doesn't get to be the shliach tzibur, because we don't really suspect him of worshiping a"z, but we penalize him for doing something that is commonly done by idol-worshipers.
Your conclusion seems to be wrong for two reasons: 1. It is not the way of avodah zarah to wear black jackets and hats (Hindus wear colored clothes, I think). 2. It is the minhag of all of Israel to wear black jackets.