This was a originally a "middus Chassidus" in the time of the gemara. As time went on, in became a minhag of klal Yisrael and gained the status of a din, because everyone took it on. As explained below, the difference is based on whether or not one runs into difficulties at work.
Talmud Bavli Kiddushin 31a, & Shabbat 118b.
Shulchan Aruch, Orach Chaim 2 (siman 6), 8 & 91, and Taz.
Rabbi Moshe Feinstein, Iggrot Moshe, Orach Chaim, vols. 1 & 4; & Choshen Mishpat, vol. 1.
Kippah: A Blessing On Your Head
From a biblical standpoint, only the Kohanim serving in the Temple
were required to cover their heads (see Exodus 28:4). Yet for many
centuries, the obligatory custom has been for Jewish men to wear a
kippah all the time, as the Code of Jewish Law says, "It is forbidden
to walk four cubits without a head covering."
Kippa why?
The wearing of a kippa (skullcap) first appears in the Talmud as an
act of piety. Another word for kippa is Yarmulke, which means "awe of
the King [G-d]" in Aramaic. This practice is codified in the Shulchan
Aruch as an obligation at the time of prayer, and as something that
one "should do" at other times. Therefore according to the Shulchan
Aruch, a head cover is a Halacha (Law) during prayer, and an important
custom at other times.
However, the Taz (mid 17th century, Eastern Europe) suggests that
although a headcovering was originally an act of piety, it gained the
status of Torah Law, due to the custom of non-Jews to remove their
caps as a sign of honor. Since the Torah prohibits Jews from "going in
the ways of non-Jews," one who does not cover his head would therefore
be in transgression of a Negative Commandment of the Torah.
Rabbi Moshe Feinstein, in his Responsa, rules that, based on the Taz,
one should be stringent. He adds, however, that there are indications
that even the Taz might agree that in America [and elsewhere] where it
is no longer the way of Gentiles to remove their head coverings as a
sign of honor - for the most part they don't even wear head coverings
at all - the prohibition against going about with an uncovered head is
no longer considered to be a Torah prohibition.
