The Gemora says that, in the Beit HaMikdash, the men and women were separated during the "Simchas Beis Hashoeiva". This was because the environment was so lax and fun and lightheaded they were afraid people would do inappropriate behavior in the Beit HaMikdash.
After the Beit HaMikdash was destroyed, this concept was further used to create separate seating in the Beit K'nesset (shul) which has some rules similar to the Beit HaMikdash.
From that point on, various communities in various places have created different rules on this topic, all using this line in the Gemora as the basis for their new separation.
However, regarding the busses, another argument is based based on the line in the Gemora, which says not to walk behind a woman. This is normally understood to mean that you don't want to be watching a woman's backend, but they apply it to say that men cannot sit behind women on the bus. On the Jerusalem light rail train, the Charedim have agreed to make the last car in the train for men only, because the distance between the cars is enough that they don't have this "issue."
This mishna is further explained in Avot D'rabbi Natan:
It is written [ibid., ibid. 6]: "None of you shall approach to any
that are near of kin to him." From this it was said one must not stay
in a separate room with any woman in a hostelry, though she be his
sister or daughter, because of public opinion. For the same reason one
must not converse with a woman in the market, not even with his wife.
For the same reason a man shall not walk behind a woman, even though
she be his wife. This was deduced from the following analogy of
expression: It is written in the passage of illegal unions, "Ye shall
not approach," and here is also written, "Thou shalt not approach,"
from which it is to be inferred that one shall not approach such
things as can cause him to sin (or cause people to talk about him).
There is a correspondance of letters between the Tzitz Eliezer and R. Shlomo Auerbach when they were in their 50s, regarding entering a bus behind a woman. (based on the statement in the Gemora not to walk behind a woman) The end result is that they agree that it is a personal manner, and some people who are not strong should avoid entering a bus behind a woman, but it cannot be a general rule.
The interesting point in the correspondence, is the fact in the 15th century, they mention that the "times have changed", and that in the days of the Gemora or the Rambam, it was rare to find a woman out in the marketplace. However, today, women are found all over the marketplace, and it's impossible to avoid walking behind a woman easily. Therefore, we are lenient on this ruling. If you are in a place where there are a few women (e.g. one or two) then, certainly, avoid walking behind them. But, if you are in a busy place, where removing yourself from behind one woman will just make you walk behind another, then stay where you are.
It is often pointed out by people today, that the Gemora and Halacha are telling a man what he should be careful in doing. But today, there are people trying to tell the woman what they should be careful in making sure the man doesn't do.