I'd suggest as follows:
The mitzvah of Korban Pesach has two components that can be done by non-kohanim - shechting it, and eating it.
The first of these most likely wouldn't be done by women. First of all, וכי אשה בעזרה מנין (Kiddushin 52b); second, it's not so simple whether women are allowed to shecht, or if yes, whether when it comes to kodshim they can do so lechatchila or not. (See Tosafos, Chullin 2a and Kiddushin 76b, and Shulchan Aruch, Yoreh De'ah 1:1.) So whether they're tamei or not would be immaterial to that part of the mitzvah. Whereas the men could, and likely would,** do their own shechting. (They may also have been aware of the rule that טומאה הותרה\דחויה בצבור, and thus argued that they ought to be able to enter the chatzer of the Mishkan for this purpose even if they're a minority, in order to be able to join with the tzibbur.)
The eating part is indeed binding on both men and women. But it's specifically about the Korban Pesach that Chazal tell us (Nazir 23a and Horios 10b) that eating it לשום אכילה גסה is wrong (and R. Yochanan goes so far as to call a person who does so a פושע). One definition of אכילה גסה is "just because he's hungry and desires it" (Mefaresh to Nazir ibid.), or "not for the sake of the mitzvah, just eating it as dessert" (Rashi to Horios ibid.). If, then, women came forward to ask what accommodation could be made for them to eat the Korban Pesach when they're tahor, they might well be accused (by Moshe, or even by Hashem if their motives weren't 100% pure) of wanting to do so לשום אכילה גסה. (Note that the men neatly sidestepped that issue; their complaint is למה נגרע לבלתי הקריב קרבן ה' במועדו, not לבלתי אכול, and Rashi there notes that they would have been satisfied if יאכל הבשר לטהורים so long as they could participate in the bringing of the korban.)
** It may be true, as DoubleAA noted in the comments, that many men wouldn't be doing the shechting anyway (i.e., that multiple households, each with their own male head, would share in one Korban Pesach). I'm not convinced of that point. But even if we grant it: each man could still be the one to shecht his group's korban (if they're able to do so even when tamei), while the women would not.