Rule #1: Don't fight about it.
First, one should never get into fights with others to do Mitzvot (Mishna Berura 53 sk 65). Regarding our case, the Chatam Sofer (YD 345) writes that if you deserve a certain spot and someone else takes it, he hasn't gained anything and you haven't lost anything (ie. you get the "credit" anyway). Plus fighting another Jew seemingly outweighs any benefit you can think to get from being Chazzan.
Rule #2: Like every Chazzan, the mourner needs to be qualified to lead the service (be able to read the words, etc.) (Mishna Berura 53 sk 60). Moreover, if the congregation wants a specific person to lead them (eg. he has a nicer voice), he has precedence over any mourner and any rule below (ShA OC 53:20).
After that, the main if not only rule is to follow the custom of the community since these matters are not formal law (Arukh haShulchan YD 376:16). It seems like a good idea for a community to formalize a policy so as to avoid any fighting.
Various customs have been recorded and they can be used as a starting point if there is no other custom. Unless otherwise noted, what follows is from the set recorded in Beiur Halakha 132.
A mourner can fall in one of five time-periods, listed in decreasing order of precedence: Shiva, Sheloshim, Yahrtzeit, last day of the 11 months, rest of the 11 months. (Some place Yahrtzeit before Sheloshim, see Arukh haShulchan ibid.) For these purposes Shiva and Sheloshim are the full 7 and 30 days, independent of partial final days or interruptions from Yom Tov.
Multiple mourners on the same level split everything and can draw lots to see who goes first.
If someone is a guest (Oreiach) in a Shul they have weaker rights than a regular member (Toshav). Thus, a Toshav who is in the same period as an Oreiach gets precedence, but the Oreiach gets to lead once during his visit (except in the case of Shiva when they split evenly, seemingly even in a Shiva house itself). If they are in different periods, the Oreiach has rights equal to one level below what he would have at home (except if the Oreiach has a Yahrtzeit when he has no rights since he should have planned his calendar better). An Oreiach's "last day" is just like any other day in the 11 months. If there are two Orechim on the same level and one of them is mourning a deceased relative who was a Toshav, he gets precedence.
A son mourning a parent has precedence over someone mourning any other relative. Some say this is when the son is in Shiva or Sheloshim, but if he is in the rest of the year, they have precedence if they are in Shiva or have Yahrtzeit (Gesher HaChayim 30:10:7). If the deceased had no living sons and a different relative is substituting, some say a grandson gets precedence over a son-in-law who gets precedence over anyone else (Sedei Chemed, Avelut 158).
Two brothers mourning for the same parent each have full rights against a third person mourning his parent, ie. they split everything three ways (Rama YD 376:4).
Some say that in determining who goes first when splitting, a Kohein has precedence over a Levi and a Levi over a Yisrael (Peri Megadim EA 53:14).