R Yisroel Reisman in his book Pathways of the Prophets asks a very similar question (p. 413) regarding the source of the dispute between Rashi and R Tam, specifically how could such a dispute have started if there was a continued tradition of wearing tefillin?
He explains there are two approaches to explain -- the second sheds light on your question.
Rabbi Asher Wassertheil in his Bircas Asher quotes Rashi and seems to imply that it is a reference to the second set of quail and not the earlier occurrence. He writes expressly:
"כלומר השליו נחשב כאן לנס חיובי"
"This means that the quails mentioned here (in Rashi) are to be regarded as a positive/obligatory miracle".
So Calev was ...
Rav Menachem Mendel Schneerson, the late Lubavitcher Rebbe asked this exact question in his Likkutei Sichos (Vol. 8, pg. 85). The answer is quite lengthy but it's a worthwhile read.
For non-Yiddish speakers, you can find an English adaptation of his answer in the Gutnick Chumash, It can be accessed easily on this Google Doc.
According to Menachem Zohari in his series of books on the sources of Rashi's commentary on Tanach, "Mekorot Rashi", in the book on Yechezkel, Rashi's sources for this commentary are:
The Talmud in Erchin 12a, as brought in the above answer by @AlBerko.
Two sections of Seder Olam Rabbah:
a. Chapter 11 (my translation):
"...and so it says &...
IIRC, the explanation goes like this:
There's no Biblical festival of R"H, there's one of blowing the Shofar on the first day of the 7th month.
On Jubilees, the main Shofar (to free the slaves, etc) is blown on the 10th day of the 7th month as it says:
"וְסָפַרְתָּ לְךָ שֶׁבַע שַׁבְּתֹת שָׁנִים שֶׁבַע שָׁנִים שֶׁבַע פְּעָמִים וְהָיוּ לְךָ יְמֵי ...