Rashi does not say that one should not join the army younger than 20, as the questioner put it. That is an interpretation read into the words of Rashi.
כל יצא צבא: מגיד שאין יוצא בצבא פחות מבן עשרים:
all who are fit to go out to the army: This informs [us] that no one went out to the army below the age of twenty.
This can be understood as an explanation of the verse, that in the army assembled by Moshe, the age at which they were selected / drafted was from 20 and up.
If you want to say that this comment of Rashi is halachically binding, and was intended as a halachic statement, and was intended for all generations, and as an instruction for someone who might submit to a draft, then you would need to point to a gemara, or a Rambam, or a Shulchan Aruch.
As it is, the Chabad translation (I believe from Judaica Press), which often lists Rashi's source, does not give any source here.