The prohibition comes from Devarim 22:5, for which Rashi provides commentary as follows. Please click on the image to view "full screen" mode.
Rashi is citing from the relevant section of the Babylonian Talmud (b. Nazir 8:1a, II.4.E [Folio 59A]), which appears as follows. Please click on the image to view "full screen" mode.
The yellow highlighted area indicates that the intent is the issue in addition to the concomitant public perception. Is there apparent intent for the woman to dress like a man, or a man like a woman?
This citation from the Talmud is also interesting because the stipulation includes that women cannot be soldiers (please note last paragraph where Jacob Neuser mistranslated the word "wore," which should be "war").
Neusner, Jacob (2011). The Babylonian Talmud: A Translation and Commentary. Peabody, MA: Hendrickson Publishers, pp. 222-223.