The gemorah says that IN THE BEIT HAMIKDASH, the Men and women were separated during the "Simcha Beis Hashoeva" because the environment was so lax and fun and lightheaded they were afraid people would do inappropriate behavior in the Beit Hamkidash.
After the destruction of the beit hamikdash, this concept was further used to create separate seating in the Beit Kinesset (shul) which has some rules similar to the Beit Hamikdash.
From that point on, various communities in various places have created different rules on this topic, all using this line in the gemorah as the basis for their new separation.
However, regarding the busses, another argument is based based on the line in the Gemora, which says not to walk behind a woman. This is normally understood to mean that you don't want to be watching a woman's backend, but they apply it to say that men can not sit behind women on the bus. On the Jeruselem light rail train, the Charedim have agreed to make the last car in the rain for men only, because the distance between the cars is enough that they don't have this "issue."
This mishna is further explains in Avot D'rabbi Natan
It is written [ibid., ibid. 6]: "None of you shall approach to any
that are near of kin to him." From this it was said one must not stay
in a separate room with any woman in a hostelry, though she be his
sister or daughter, because of public opinion. For the same reason one
must not converse with a woman in the market, not even with his wife.
For the same reason a man shall not walk behind a woman, even though
she be his wife. This was deduced from the following analogy of
expression: It is written in the passage of illegal unions, "Ye shall
not approach," and here is also written, "Thou shalt not approach,"
from which it is to be inferred that one shall not approach such
things as can cause him to sin (or cause people to talk about him).
There is a correspondance of letters between the Tzitz Eliezer and R. Shlomo Aurbach when they were in their 50s regarding entering a bus behind a woman. (based on the statement in the gemora not to walk behind a woman) The end result is that they agree that it is a personal manner, and some people who are not strong should avoid entering a bus behind a woman, but it can not be a general rule.
The interesting point in the correspondence, is the fact in the 15th century, they mention that the "times have changed", and that in the days of the Gemora or the Rambam, it was rare to find a woman out in the market place. However "today", women are found all over the marketplace, and it's impossible to avoid walking behind a woman easily. Therefore, we are lenient on this ruling. If you are in a place where there are a few woman (say 1 or 2) then certainly, avoid walking behind them.. but if you are in a busy place today where removing yourself from behind one woman, will just make you walk behind another, then stay where you are.
It is often pointed out by people today, that the Gemorah and Halacha is telling a man what he should be careful in doing. But today, there are people trying to tell the woman what they should be careful in making sure the man does not do.