Medieval scholars generally consider the letter authentic. The issue is complicated by the fact that there are texts scholars doubt the Rambam wrote, including a work that incorporates this letter. As Joshua Prawer writes in his work The History of the Jews in the Latin Kingdom of Jerusalem:
The text was found at the beginning of a Commentary on Tractate Rosh ha-Shanah wrongly assigned
to Maimonides, but there is no reason to doubt the veracity and authenticity of the biographical note. It
was copied by one R. Samuel b. Abraham Secal in Acre. The copyist is probably a man from a well-known
wealthy family in Narbonne in the last quarter of the 13th cent. (p. 142 note 37)
Similarly, Jewish scholar David Kraemer writes:
Although the letter is quoted in a commentary erroneously attributed to him, it is accepted as genuine by scholars. (Jerusalem, 1000–1400: Every People Under Heaven, p. 302, n. 2)
If you want to see what the Secal copy of Rambam's letter looked like, here are the pages of the manuscript, archived in the Bibliothèque Nationale de France (Manuscript 336):

As chortkov2 mentioned in a note, the letter was also replicated in Sefer Haredim. In terms of modern rabbinic scholars, Rav Yitzhak Sheilat, who edited a critical edition of the Rambam's letters (Ma'aleh Adumim), similarly affirmed its reliability by consulting other copies of the letter. After quoting the last part of the text, that it was copied from the Nagid R. David [the Rambam's great-grandson] in Akko, R. Sheilat writes:
ונראה כי בעיקרה אמיתית היא , אך נתקצרה מלשונו המקורית של ר' דוד , ואף נפלו בה שיבושים
סביר , איפוא , להניח שעדות כ"י פריס נכונה , והיה בידי ר' דוד בעכו פירוש ראש השנה , וכן
הרשימה האוטוביוגרפית , בכתב ידו של הרמב"ם
It appears that this is essentially genuine, but was abridged from the original language of R. David, with some corruptions that entered. It is reasonable, therefore, to assume that the testimony of the Paris manuscript is correct, and it was from the hand of R. David in Akko in the commentary on Rosh Hashanah, so that the autobiographical section was by the Rambam (p. 224-225)
Whether or not the Rambam ascended to the Temple Mount depends on how you read his words of entering "the great holy house" in Jerusalem. Thus, it's not about the letter's authenticity as much as it is a question of whether the Rambam would go against his own writings on the subject (an issue that requires its own question). This is why scholars like Joel L. Kraemer, who agrees he visited Jerusalem, doesn't think he went up:
It was difficult for Jews and Muslims, though not impossible, to get access to the Temple Mount. Maimonides would most likely have prayed outside it at the Western Wall. At the time, the Church of St. Giles was located nearby. No matter where he looked, he would have seen one or more of the many churches established by the Crusaders in Jerusalem. Jews also prayed on the Mount of Olives, facing the Golden Gate.
The Crusader historian Joshua Prawer understood Maimonides’
statement—“I entered the great and holy house and prayed in it”—to
mean that he entered the Temple Mount and even the Dome of the Rock.
But there is no mention of the Dome of the Rock, and it is doubtful
that he would have described it as “the great and holy house.” Prawer
finds that this raises numerous questions. For example, Maimonides
ruled against Jews entering the holy precinct because of their ritual
impurity. The sites that had a special importance were the Mount of
Olives and the area surrounding the Temple Mount, including segments
of walls and specific gates. Maimonides’ opinion was rejected by Rabbi
Abraham ben David in his comment on the ruling, and the halakhic
discussion continues to our own day. (Maimondes: The Life and World of One of Civilization's Greatest Minds , p. 137-138 )