There are many unsatisfactory theories proposed (see below), but I think, there is a good psychological reason for such a phenomenon. A person may understand if they ever went to big mixed gender parties or studied in mixed gender schools. Men and women tend to think differently: men think sequentially while women think in parallel with many thoughts at the same time. Men easier pick the main idea, women pay attention to details. When many men get together they keep independence, and their plurality/group with multiple parallel thoughts thinks as a female! When many women get together, the opposite occurs. They all act the same. For example, when a guy approaches a girl from a group and gets rejected, all girls in her group start rejecting him. Thus, a group of women thinks like a male!
The book Joshua Blau "Phonology and Morphology of Biblical Hebrew" on pp. 279 - 280 discusses four different hypotheses as a possible answer to the question. The book notes that this is a stark contradistinction to other parts of speech, which characterizes Semitic languages in general (except where later development has blured it), which has not yet been explained adequately. The four suggested answers adduce to three theories not because they are substantiated, but because they attest to one of the fundamental weaknesses inherent of historical linguistics: logically built theories, ingeniously conjectured and reflecting profound knowledge of the subject, very often remain beautifu hypotheses, without possibility of verification.
For pedagogical reasons I am also going to bring an oldest fifth theory from Gesenius' Hebrew Grammar pp. 286-287 and start with it. The first three theories assume that numerals were nouns first and afterwards evolved into a separate part of speech.
Explanation 0. Brockelmann, Sem. Sprachwiss. p. 116 ff; Grundriss, i. 484 ff.
The formation of the cardinal numbers from 3 to 10 (but not 1 and 2) has the peculiarity that numerals connected with a masculine substantive take the feminine form, and those with a feminine substantive take the masculine form. The common explanation of this strange phenomenon used to be that the primary form of the numeral was an abstract noun in the feminine, i.e. ending with Heh (ה) or Tav (ת). This was originally attached in the construct state to the word qualified, then came to be also used in apposition to it, and finally was placed after it like an adjective. The consequence of the appositional, and finally adjectival, construction was, that for numerals connected with feminine nouns a special shorter form came to be used, whilst the original forms, with the abstract feminine ending, were used in connection with masculine nouns, after as well as before them.
On this view the historical process would have been that originally the abstract numerals (like Latin trias, decas, Greek πεντάς, δεκάς, &c.) were placed in the constr. st. before masculines and feminines alike, e.g. שְׁל֫שֶׁת בָּנִים trias filiorum, עֲשֶׂ֫רֶת נָשִׁים decas mulierum. A trace of this earlier usage was seen in the examples, like שְׁל֫שֶׁת נָשִׁים. Further, it was possible to say שְׁלשָׁה בָנִים trias, sc. filii, as well as בָּנִים שְׁלשָׁה filii, trias. From this second appositional construction it was only a step to the treatment of the abstract numeral as an adjective, filii tres. Similarly the subsequently shortened forms of the abstract numeral, which were used in connexion with feminines, might stand either in the construct state before, or in apposition before or after the word numbered, thus שְׁלשׁ בָּנוֹת trias filiarum, or שָׁלשׁ בָּנוֹת trias, sc. filiae, or בָּנוֹת שָׁלשׁ filiae, trias, or adjectivally filiae tres.
Explanation 1. Hermann Solomon Reckendorf, 1898-98 "Die syntaktischen Verhaeltnisse des Arabischen" 2 vols, Brill Publishers, Leiden pp. 265-87 (also noted in Gessenius' grammar). I am combining Gessenius description of it as it is clearer and Blau's as it is more modern:
He also considers that the earliest forms were abstract numerals which were placed in the construct state before the noun numbered, the latter depending on them in the genitive. The original form, however, of the abstract numerals from 3 to 9 is not the feminine, but the masculine, used for both genders, as it still is in the tens, 20, 30, &c. In other words the form had no ending (e.g. שָׁלׂשׁ), whereas BHeb עֶשׂרֵה, Ugarit 'ShRH, Arab 'AShRaT(a)- denote decade. Accordingly שָׁלׂשׁ-עֶשׂרֵה means three of the decade, i.e. thirteen and the original relation between these two numerals was that of a construct noun and a governed noun. The feminine abstract numeral was first distinguished by a special form in the numbers from 13 to 19 when connected with masculines, and this distinction was afterwards extended to the numbers from 3 to 10. This explanation does not affect the view stated above that the appositional and adjectival use of the abstract numerals was only adopted later in addition to their use in the genitive construction.
The differentiation of the numerals (originally of common gender) into masculine and feminine forms in the second decade, was occasioned, according to Reckendorf, by the use of the עֶשְׂרֵה form in compounds, which felt to be abstract feminine due to ending ה, and similarly Ugaritic -h, Arabic -at. So long as it was felt that שְׁלשׁ-עֶשְׂרֵה simply meant the three of the decade, the gender of the noun numbered made no difference. When, however, the consciousness of this meaning became weakened and the combination of units and tens came to be felt as a copulative rather than a genitive relation, it seemed suitable to connect only feminine nouns with the feminine form עֶשְׂרֵה. New forms were therefore invented, both of the units and the tens, for use with masculine nouns, by polariry, like שְׁלׂשָה-עָשָׂר. The former, however, no longer had the form of the construct but of the absolute state, clearly showing that the consciousness of the original syntactical relation in שְׁלשׁ-עֶשְׂרֵה, &c., was lost. On the other hand, after the extension of these new formations to the first decade (single numbers), the new feminine forms readily came to be used also in the genitive construction (and therefore in the construct state) on the analogy of the earlier masculine forms.
Explanation 2. Jakob Barth, 1907-11: Sprachwissenschaftliche Untersuchungen zum Semitischen 2, pp. 1-17.
He positted that the cardinals three through ten had a zero ending (i.e. were masculine). In his opinion the femminine forms שְׁלׂשָׁה and שְׁלׂשֶׁת, etc., did not originally terminate in feminine ending, but in the masculine pronomial suffix -tu, as preseved in the Ge'ez (shalastu). Later, shalastu, etc. although it refered to masculine nouns was felt as terminating in the feminine suffix, as if feminine numbers refered to masculine nouns. In Ge'ez the feminine counterpart of the masculine pronomial suffix is -ti is only preserved in the numeral two: k'l'etti in contrats to masculine k'l'ettu.
Explanation 3. Hans Bauer 1912 paper, "Noch einmal die semitischen Zahlwoerter", pp. 267-70
The numbers like שְׁלׂשָׁה and שְׁלׂשֶׁת do not terminate in the feminine ending. In his opinion the original ending suffix was -tu, however it was not a pronomial element, but was related to Arabic taww - single. In Hebrew we have תּוּ - again, furthermore, i.e. another single as a possible analogue. Therefore, there were two ways to denote, e.g. five ships (אֳנִי being fleet, אֳנִיָּה its momen unitialis):
חָמֵש + תַוו + אֳנִי, literally five + single + fleet, or
חָמֵש + אֳנִי + תַוו, literally five + fleet + single
i.e. Bauer also posits that singular male noun was used to denote plurality. The first form חָמֵש + תַוו + אֳנִי got shortened to חֳמֵשֶת אֳנִי with חֳמֵשֶת referring to masculine אֳנִי. At the same time, the second form חָמֵש + אֳנִי + תַוו got shortened to חָמֵש אֳנִיָּה with חָמֵש referring to feminine noun אֳנִיָּה.
Bauer similarly interprets the origin of the feminine suffix ת in nouns.
Explanation 4. Elmar Ternes, 2002 paper, Entgegengesetzte Genuszuweisung bei Numeralia im Semitischen: einige grammatiktheoretische und typologische Überlegungen, 719 - 36. Some other linguists also believe that this is the case.
They believe that this was an archaic Proto-Semitic feature which is a residue of a more wide-spread phenomenon. No explanation is given.