There are many reasons why Muhammad could not have been a true prophet, according to Jewish belief. I'll summarize at the top of this answer and then drill down into particulars.
Because of Bilaam's wickedness, as emblematic of the wickedness of gentile prophets, God removed prophecy from the gentiles (Midrash Tanchuma, Balak, siman 1).
Muhammad was born too late. Prophecy was taken from the world at the beginning of the second Temple period. (Talmud Yomah 9, Sotah 48)
The set of gentile prophets appears to be a closed set, limited to seven at most (Bava Batra 15b). This reflects the idea that prophecy from God is generally restricted to the people of Israel.
Muhammad founded a religion which differs greatly in theology and matters of Biblical history from that established by the Torah. Such a contradiction would render a prophet a false prophet. For instance, who was bound by Abraham? Was it Ishmael or Isaac?
Now to examine each of these in turn, in greater detail.
1
According to Midrash Tanchuma, prophecy was explicitly removed from the gentiles in the time of Bilaam. The other gentile prophets, including Iyov, lived about the same time.
This midrash reads:
וכל גדולה שנטלו ישראל, את מוצא שנטלו האומות כיוצא בה.
העמיד משה לישראל שהיה מדבר עמו כל זמן שירצה.
העמיד להם בלעם, מדבר עמו כל זמן שירצה.
ראה מה בין נביאי ישראל לנביאי האומות.
נביאי ישראל מזהירין את האומות על העבירות.
וכן הוא אומר: נביא לגוים נתתיך.
ונביאים שהעמיד מן האומות, נותנים פרצה לאבד את הבריות מן העולם הבא.
ולא עוד, אלא כל הנביאים היו במידת רחמים על ישראל ועל אומות העולם.
שכן ישעיה אומר: על כן מעי למואב ככנור יהמו וגו' (ישע' טז יא).
וכן יחזקאל אומר: בן אדם שא על צור קינה (יחז' כז ב).
ונביאי אומות העולם, היו במידת אכזריות, שזה עמד לעקור אומה שלמה חנם על לא דבר. לכך נכתבה פרשת בלעם, להודיע למה סלק הקדוש ברוך הוא רוח הקדש מאומות העולם, שזה עמד מהם, וראה מה עשה:
My translation:
And every greatness you find that Israel took, the nations took likewise. He established Moshe for Israel, who spoke with him any time he wished. And He established for them [the gentiles] Bilaam, who spoke with him any time he wished.
See the difference between the prophets of Israel and the gentile prophets. The Israelite prophets warn the gentile nations about sins. And so is said [regarding Jeremiah] 'I have set you as a prophet for the nations. And the prophets He established from the gentile nations, they placed a breach to remove people from the world to come.
And not only that, but all the [Israelite] prophets worked via the trait of mercy upon both Israel and the nations of the world, for so Isaiah says, 'therefore my bowels for Moav vibrate like a harp, etc.' (Isaiah 16:11) And the nations of the world acted with a trait of cruelty, for this one [Bilaam] arose to uproot an entire nation for absolutely nothing.
Therefore the parasha of Bilaam was written [in the Torah], to inform why the Holy One, Blessed Be He removed the holy spirit [prophecy] from the nations of the world, for this one arose from them, and see what he did!
2
Muhammad, like
Minever Cheevy, was born too late. According the the Talmud (Sotah 48b), at a specific point in time, prophecy was removed from the Israelites.
In Sotah 48b:
Our Rabbis have taught: When Haggai, Zechariah and Malachi died, the Holy Spirit10 departed from Israel; nevertheless they made use of the Bath Kol.
One can draw a kal vachomer (a fortiori), that if it was removed from Israel, surely it was removed from outside Israel.
In Bava Batra 12b, this is stated about prophets in general:
R. Johanan said: Since the Temple was destroyed, prophecy has been taken from prophets and given to fools and children.
3
The list of gentile prophets appears to be a closed set of a maximum of seven. These were all people in the generation of Bilaam. From Bava Batra 15b:
Seven prophets prophesied to the heathen, namely, Balaam and his father, Job, Eliphaz the Temanite, Bildad the Shuhite, Zophar the Naamathite, and Elihu the son of Barachel the Buzite.
Bilaam and his father (Beor) lived approximately the same time, and the other characters are from the book of Job, who is chronologically often placed at the same time at Bilaam. The Talmud continues to specify that many some of these were not gentiles, but just had prophesied toward the gentiles. This includes Job, which strips one of your two examples from your list. Another rabbinic position is that the entire book of Job is fictional wisdom literature. While Job the person, and his righteousness, were historical, the events depicted in the book of Job did not occur.
Vayikra Rabba 1:13 also lists many restrictions on gentile prophecy, in terms of its status. And one such restriction is that it ended with the erection of the Tabernacle (Mishkan):
אמר רבי יצחק:
עד שלא הוקם המשכן היתה נבואה מצויה באומות העולם. משהוקם המשכן נסתלקה מביניהם, שנאמר: (שיר ג)אחזתיו ולא ארפנו.
אמרו לו: הרי בלעם מתנבא?!
אמר להן: לטובתן של ישראל נתנבא.
(במדבר כג) מי מנה עפר יעקב.
(שם ) לא הביט און ביעקב.
(שם) כי לא נחש ביעקב מה טובו אוהליך יעקב.
(שם כד) דרך כוכב מיעקב.
(שם) וירד מיעקב.
Bilaam seems a counterexample but it explains why he was the exception, that he prophesied for the benefit of Israel.
The general trend and assumption of Rabbinic sources then seems to be that even in their days, gentile prophecy no longer existed.
4
Muhammad founded a religion which differs greatly in theology and matters of Biblical history from that established by the Torah.
Such a contradiction would render a prophet a false prophet. Indeed, some Biblical books, by Israelite authors, were going to be tossed out of the Biblical canon for contradicting the Torah. For instance, the Qur'an has Abraham binding Ishmael rather than Isaac:
In the tale of binding (surah 37:99-110) Muhammad identified the son who was to be sacrificed as Ishmael and, indeed, the opinion of the traditionalists were also divided on this subject.
This is not the place to debate differences between Jewish and Muslim theology, and between Biblical and Quranic history. But there is certainly enough for one to argue that they differ.
For all these four reasons, at least, Jews would not have believed that Muhammad was a true prophet.