For someone who commits an act of "mesirah" (giving over a Jewish person to law enforcement) What punishment if any does such a person (the "moser") receive? Or could/should he receive? Where is the punishment stated? Is being put into cherem a possible punishment?
In general, the concept of "mesirah" is not "giving over a Jewish person to law enforcement" as much as giving an innocent person over to a ruthless, corrupt, and brutal power. See e.g. Rabbi Yitzchok Grossman:
The topic of Mesirah in the modern era is a very fraught area of Halachah. On the one hand, Shas and Poskim have traditionally condemned Mosrim in the most drastic terms: they are included (along with apikorsim and those who have sinned and caused others to sin) in the list of those who suffer for an extended duration in Hell,1 are worse than apostates,2 and cannot (at least according to some opinions) ritually slaughter animals,3 or write Sifre Torah4 or Gittin.5 And, of course, they may be killed “anywhere, even in contemporary times” after being duly warned, if they are threatening to inform. But on the other hand, it is clear that much of the Halachic literature on the topic was written in the context of regimes that were vastly more ruthless, brutal and unprincipled than modern Western democracies, and of Jewish communities that had much more autonomy and capability of internal regulation than today's post-Enlightenment diffuse and acculturated ones.
See also Rav Herschel Schachter, shlita:
If the non-Jewish governmental authorities know that one Jew is concealing information about another Jew in order to save him from punishment, the Shulchan Aruch (Choshen Mishpat 28:3) considers this a situation of chilul Hashem. Similarly, for many generations it was the practice that if a gneiva had occurred, and suspicion had fallen on the Jewish community, rather than allow that suspicion to hover over the entire community, the roshei hakehol, with the permission of the rabbonim, would inform the non-Jewish authorities who might possibly be the real ganav (Be'er Hagola, Choshen Mishpat 388:12).
Even if one is guilty of a crime and deserves a punishment according to the laws of the land, but due to anti-semetic attitudes he will probably suffer more than if he were a non-Jew; or, the (state) prison conditions are such that he will suffer at the hands of the other inmates (or at the hands of the guards) in a manner that is not proscribed by law, then turning the offender in would constitute mesirah, since his added suffering will be shelo kadin. However, mesirah is permitted in situations where one is a public menace (see Shach to Choshen Mishpat 388, 59), or if one is physically or psychologically harming another individual (for example, in instances of sexual abuse of children, students, campers etc., or spousal abuse) (see Shach to Choshen Mishpat ibid, 45).
The Jewish community does not have the ability to investigate these types of cases. Wherever there are raglayim ladavar that there seems to be a problem, the proper government agencies should be contacted to investigate.
Just as in other areas of halachah, one should consult a competent moreh horaah when faced with such a shayla. Just because one is knowledgeable in Yoreh Deah vol. I or one delivers a good pilpul shiur on sugyos in Nashim or Nezikin, it does not necessarily follow that that individual will be qualified to pasken on hilchos mesirah - lehakel or lehachmir.
That said, giving an innocent person over to a ruthless, corrupt, and brutal authority (such as the pagan governments that existed at the time of the Talmud) would confer upon the culprit the status of a rodef, murderous pursuer where we save the victim by any means necessary, including with the life of the pursuer (see e.g. Bava Kama 117a and Teshuvoth HaRosh 17:1).
Finally, the halacha is that killing a rodef when a less violent means could have accomplished the same end (e.g. yachol l'hatzilo b'echad me'eivarav "he could have saved him by [injuring] one of the [pursuer's] limbs") constitutes full murder. The equation then of a moser to a rodef suggests that if indeed a moser could be stopped with just a cherem (ostracization), violence would be prohibited by halacha.
See also JLaw's article on the topic.
1 Rosh Ha'Shanah 17a, and see Shach CM 388:53.
2 Shach ibid. and #62.
3 Shach ibid., citing Shulchan Aruch YD 2:9.
4 Shach ibid., citing Shulchan Aruch YD 281:3.
5 Even Ha'Ezer Siman 123: Get Pashut #7 s.v. le'inyan mumar u'malshin, and Yeshuos Ya'akov (short) #9, cited in Pis'chei Teshuvah #4.
Shulchan aruch choshen mishpat 388 is about these laws
If he gave over money: he needs to repay.
if he gave over money or people he does not have a part in the world to come If he gave over 3 times money or people: we try to kill him passively.
If he will do it: we can use any means to stop him even to kill him (is a mitszva, first one to come gets to do it) even in our times (shulchan aruch said this) (but not after he did it, if he is not going to do it again)
(There are exception when it is permitted to give over a Jew).
(If you want a sposific law translation/details ask for it in the comments).
It seems that a Bais din can put him in chairem