As @Ezra correctly notes in his answer, The pasuk clearly says (Bamidbar 13:20):
What is the soil like is it fat or lean? Are there any trees in it or not? You shall be courageous and take from the fruit of the land." It was the season when the first grapes begin to ripen.
I discuss a related issue (why point to the ripening of the grapes if they also bring pomegranates and figs) in a blog post, here, where I point to a chart from Agriculture in Iron Age Israel (Oded Borowski, 1987, page 7), which tries to fix the seasons in Ancient Israel on the basis of harvesting in modern Israel, under the theory that the climate in the same geographical location would be the same across eras:
This would address @DoubleAA's question / comment on @Ezra's answer,
If none were in season, then why did they bring pomegranates with the grapes?
As you can see from the chart, figs and pomegranates were not yet ripe, but were almost there, and could be brought by the scouts to a waiting Hebrew populace eager to sample the fruits of Israel.
Olives only ripened months later. And wheat, barley and oats had already ripened and been harvested months earlier.