According to the Ra'avad, Har Habayis (Temple Mount) does not have Kedusha nowadays, so there would be no halachik issue with visiting any part of it. However, most poskim assume like the Rambam that Har Habayis retains its kedusha even though the Beis haMikdash has been destroyed.
This means people can only visit areas that are allowed based on their level of taharah/tu'mah. Nowadays, we assume everyone is tamei meis, so md'oraysa no one can go into the azara (Temple Courtyard), and md'rabanan one cannot go past the cheil (a gate the encircled both the Azara and Ezras Nashim).
A person with a tumah yotzei megufo cannot go into Har haBayis. He therefore needs to immerse in a kosher mikvah beforehand, and he can then go in any area outside the cheil.
No one claims there is a halachik problem with someone who toveled from going on har haBayis outside the cheil. The issue they raise is that perhaps we don't know where the Mikdash and Azara were and one may accidentally go into it. However, there is a large area from the outside of har habayis to the azara, so this doesn't explain why one couldn't go a few feet into har habayis. (There is also both a tradition and other evidence that the Dome of the Rock is where the Kodesh haK'doshim was.)
Many feel one shouldn't go on har habayis because Jews didn't go in the previous few centuries. However, this was because Jews were banned from going on it. Previously, when Jews were allowed to go on Har Habyis, many did so, such as the Rambam and the Radvaz.
While many rabbis say one should not go on har haBayis, the reasons are not so clear-cut, and the basic halacha does allow it. Some Rabbis feel its important to go on har haBayis, since it shows the Jews still care about it, and it is an opportunity to keep the mitzvah of mora'ah (reverence) for the mikdash, and the mitzvah of distinguishing between the permitted and forbidden areas.
It is important that one who goes on har haBayis knows the basic halachos of tumah/taharah, which areas one can go, and the halachos of mora'ah (e.g. no shoes allowed).
See also:
http://www.ou.org/torah/article/tzarich_iyun_har_habayit
http://templemountcenter.com/Aliya_English.html