Taking it for granted that the issue of "male attire" is ultimately not the source for the prohibitions, a stance I believe is justifiable even if not un-debatable, there remains a significant opposition to women wearing slacks on the grounds of modesty.
Although most noted for taking the hard-line position that wearing slacks violates the prohibition of wearing male attire, the Minchas Yitzchak 2:108 rules in very strong language that even if they did not constitute male garments such slacks were forbidden because they are immodest.
Similarly the Tzitz Eliezer 11:62 writes a lengthy teshuvah (which I hope to look at in more detail) in which he rules that pants are male attire, but also that they are clearly immodest.
So too with the Shevet HaLevi 6:118(2) who see's there as being an issue of pritzus in addition to, or at the root of, pants being male attire.
For all of these poskim, even if we were to discount the issue of pants being a male garment we are left with them being immodest garments.
And while the Yaskil Avdi 5 YD20 rules that the prohibition of wearing a male garment doesn't apply, he rules they should be prohibited because they are immodest.
So even if we do not take into account the issue of pants being a "male garment" we see significant objections to women wearing slacks in the poskim because they are not modest.