Rashi answers with an example:
Someone was selling a donkey. The buyer wanted to test the man to see whether the animal was in good health. He asked him for a test drive. The seller agreed, and now the buyer knew that it was in good health, but he wanted to really make sure. He then asked “How about the mountains? Can I take it there?” The seller quickly answered Yes. The buyer now understood that if the seller was willing to allow the donkey to be tested in such a strenuous manner, it must be a healthy animal, so he bought it.
This should also have been the response of Klal Yisroel. You want to know about the land? Fine, Hashem will allow you to spy it out. This was good in Moshe’s eyes: now take his word, and make the purchase. However, they did not. They wanted to see it for themselves. And hence Moshe chastised them.