The m'raglim were the greatest of the yidden. Even Joshua and Caleb had to be 'prayed' for or pray themselves they shouldnt join them. What was so difficult to bring about the 'correct' result. And why did then Joshua send spies seeing what disastrous results that could happen.
Rabbi Frand points out that they were not originally reshaim. They were righteous men who wound up rationalizing that it would be better for Bnai Yisrael to remain in the desert, living a miraculous life. Once they started rationalizing, they started down a slippery slope that led to their bringing a "bad report". Another point is that they allowed themselves to be panicked as it says in 13:33
לג וְשָׁם רָאִינוּ, אֶת-הַנְּפִילִים בְּנֵי עֲנָק--מִן-הַנְּפִלִים; וַנְּהִי בְעֵינֵינוּ כַּחֲגָבִים, וְכֵן הָיִינוּ בְּעֵינֵיהֶם.
33 And there we saw the Nephilim, the sons of Anak, who come of the Nephilim; and we were in our own sight as grasshoppers, and so we were in their sight.'
Rav Hirsch at the beginning of the parsha points out that they were not the official leaders of the tribes, but people who were regarded with respect within the individual tribes. After all, the actual nesi'im were considered "too old" to actually go on such a trip. Thus, they sent "young" men who would be more able to deal with the trip and also would be able to find the areas of the land most appropriate for the each tribe.
In answer to your question about how Yehoshua could send spies lies in the difference between the two missions. The group that Moshe sent was more like a congressional junket and the meraglim were more like the chief staffers. Thus, the meraglim could not sneak around and attempt to find out things. The stood out as Bnai Yisrael and were noticed immediately. They came back and delivered their "report" in public as well as inserting their own interpretations. Had they just given "just the facts" the people would not have panicked and Moshe Rabbeinu could have sent them in with no problems.
Calev and Yehoshua were under extreme pressure to join in on the consensus so as to issue the "compromise" report. It would have been like some of the congressional committes that generate a report for the congress. It is very hard to attempt to maintain the minority report. Consider how Obamacare passed into law.
Yehoshua sent Kalev and Pinchas who hat already proven themselves able to resist the pressure and maintain the truth. He sent them secretly (undercover) and they reported to him alone. This would be like a CIA mission reporting back to the head of the agency only or like an OSS mission to Europe during the war. They reported the exact facts, recruited Rachav and made sure that no-one else knew what was being done.
Yehoshua sent Kalev and Pinchas in the same way that an army sends scouts ahead (Just as the tribes settling the East Bank went ahead as the initial scouts and front lines). That is, he knew that Banai Yisrael were supposed to fight "b'derech tevah" (according to the normal way of fighting). Thus he sent professionals ahead of the army to determine the best path, to see if the enemy was waiting with troops, and were to attack first. The cloud and the fire would no longer be available to show were to go. He already knew that Hashem would help them, but he had to show hishtadlus.
The reason and method behind Kalev and Pinchas was completely different from that of the meraglim. That is why I compared them to the OSS missions sent to Europe during the war. The decision to cross and attack had already been made. The report (to Yehoshua alone) would not change that. It would only help determine the methodology that would be used after they crossed and were moving to conquer the land.
We recently mentioned the opinion of the Zohar that the problem of the Meraglim was that they fell victim to the disease of "negius" [conflict of interest]. They were afraid that when they entered the Land of Israel, they would lose their respective positions of prominence.
The Chidushei haRi"m suggests that the problem the Meraglim had with going into Eretz Yisrael was due to the very fact that they were righteous individuals. Because they were Tzadikim, they loved and longed for the type of existence that they experienced in the Wilderness.
The Meraglim, fearing the impending dramatic spiritual decline from the heavenly existence of the Wilderness era to the materialistic existence of having their own land, opted not to enter the land.
Where, then did they go wrong? The ultimate answer is that the Almighty does not want us to live that way. This is not the life of a Jew. The life of a Jew is to live a life where he has to worry about making a living and deal with all the physical requirements and needs of this world -- while at the same time remaining a holy person.
This is the name of the game. This is what Judaism is all about. We are not like other religions who believe that we cannot bridge the worlds of spirituality and physicality.
The Kotzker Rebbe emphasized the first two words in the pasuk that begins, "Anshei Kodesh tiheyun Li" [Men of Holiness you shall be to Me - Shmos 22:30]. HaShem wants MEN of holiness, not just angels of holiness. G-d wants Man with all his evil inclinations and lusts and laziness. G-d desires to see such a creature become holy. Holy MEN, not holy angels!
Achieving spirituality in a physical world is a difficult task. The Meraglim legitimately had what to fear, but that is how the Almighty wanted life to be. He put us here and made us physical beings but our life's work is to try to transform the mundane into the holy -- to take the wine and make it into Nesachim, to take the bread and make it into Challah, and to take the wool and to turn it into Tzitzis.
My answer goes a long an altogether different track. The previous answers do not fully answer my questions. The yidden in the desert where on the highest madrega a dor deiah. These meraglim were 'n'siim' 'anshai shem' in other words the top. Hashem is trying to tell us something here like everything in t'nach it is not just a story. We find even though they were so 'high', at every 'little' step they faltered. They made the golden calf and wanted every time to go back to Egypt. Even after forty years when Aaron died they didnt want to go on and went back seven 'journeys'. The lesson is that a person has to work for his madregos. They were given it on a 'plate'. They had manner to eat. True one can get very high but the slightest ill wind throws them off balance. This applies to even the greatest of them. Joshua sent the spies who basically all they did was to find out the 'mindset' of the people there. He was also scared that the yidden would turn back unless they knew that the people there were scared of them.
The spies that Moshe Rabbeinu sent, although he wasn't aware, were sent with bad intentions. The people didn't trust that the land was great, but to Moshe they told a different story. Therefore some say (I do not recall who, right now) that since the Halacha is that a messenger has the status of the one who sent him, they got evil thoughts. On the other hand, Yehoshua initiated the idea of the spies that he sent.
Additionally, these spies were sent with questions to answer. Moshe Rabbeinu, being sure of the land's greatness, told them to report about the nature of the land. They got it into their head to be true reporters and to objectively judge the land without any preconceived notions from what Hashem said about it.
Yehoshua and Kalev, through their righteousness saw everything through the real truth, that they knew to be absolutely true. It turns out that these two were the only ones to see things correctly, that the land is fruitful, and that the inhabitants are healthy, and that Hashem kept them preoccupied. On the other hand, those who threw away truth in the name of 'objective truth' which is actually unguided truth saw a very narrow view and fit all else into that view.
Yehoshua sent tried and tested men. Not just were they great in insight and purity, but they were proven to be fully loyal to Hashem. What was holy to Hashem was holy to them and what was detested by Hashem was hated by them.
When these two spied in Yericho, their whole experience was to be chased into hiding and escape, while striking a deal. And yet, they saw much past that. They reported that all the people are terrified of them.
The Medrash calls the 10 spies, fools. This is because they didn't see things correctly.
To summarize the above, there are two distinctions between those of the Midbar and those of Yehoshua. First is the intention of the actual senders. Second is that Kalev and Pinchas were trusted, tried, and tested.
But a very basic answer is that just because someone, or even a group, fell through with something, that doesn't mean it is an irresistible pitfall. Moshe Rabbeinu wasn't afraid to go up for the second forty days.
Moreover, the Meraglim didn't go unnoticed. They had to stick around an extra 38 years because of that. Believe me the lesson was learned. Moshe Rabbeinu brought it up at every opportunity. It wasn't going to happen again.