The Haggada speaks about 4 sons, the second of which is called "wicked". He distances himself from G-d's commandment, and in turn is told that if he had been in Egypt he would not have been redeemed. How can we say that? How do we know? How can we make such calculations (Cheshbonos)?
Well, we do have the precedent that four-fifths (or even more) of the Jews in fact did not want to leave Egypt and serve Hashem, and died during the Plague of Darkness (Rashi to Exodus 13:18, quoting from Midrashim Mechilta and/or Tanchuma). So based on the attitude that this second son shows towards the mitzvos of Pesach, there is indeed a good likelihood that he would have been in that category, G-d forbid.
That said, the Lubavitcher Rebbe zt"l explains (here, here and in other places) that in this rebuke there is actually an implied positive statement. "Had you been there, in Egypt, where being part of the Jewish People was a matter of one's choice," we tell him, "then you might well have remained where you were. But you are living now, after the giving of the Torah, when Hashem chose us as His people, and this remains true no matter what choices you make. So it is inevitable that you will do teshuvah and be redeemed, along with all other Jews, from our current exile, with the coming of Moshiach - may it be soon."
The Rasha says mah haavodah hazot lachem, "what is this offering to you?" Thus, he has no portion in the korban, as the haggadah interprets it "to you and not to him". Meanwhile, in makkat bechorot, Hashem only skipped over the houses of those who had offered the paschal offering.
He would not have made it to the geulah.
Similar to @josh's answer, a thought of my own.
G-d's message to Pharoah was "Let my people go so they may serve me"
The 4/5th of the Jews who died in Egypt were Jews who did not want to leave Egypt.
So the Rasha, who excludes himself from the serving G-d, would not have wanted to leave Egypt, since he had no interest in serving G-d. If he had been there, he would not have been redeemed.
Artscroll's The Answer Is... brings several answers to this question:
MA'ASEH NISSIM: The Haggadah does not say that he would not have left Egypt, but rather that he would not have been redeemed. The word geulah, redemption, can refer to acquisition. This implies that Hashem would not have been willing to "acquire" the wicked son as a servant even if he would have left Egypt.
BIRKAS HASHIR: In Egypt, even the wicked people circumcised themselves and partook of the korban Pesach (after their initial refusal to do so). The wicked son, on the other hand, would not have been willing to participate in the korban Pesach or the mitzvah of circumcision, and therefore he would not have been redeemed.
KSAV SOFER: Even though the Jews had fallen to the forty-ninth level of impurity, they still had respect for their elders and would not have dared to be insolent to their leaders. They were therefore worthy of redemption. Had this wicked son been there, however, and displayed the same insolence that he shows at the Seder, he would not have been worthy of redemption.
ALTER OF KELM: The wicked son might think that he could have simply followed the masses and been redeemed along with the Jews when they left Egypt despite his wickedness. We therefore tell him that Hashem watches over each individual and knows every person's thoughts. Had the wicked son been in Egypt, Hashem would have distinguished him from the masses and he would not have been redeemed.
SHA'AR BAS RABBIM: Hashem commanded the Jews to mark the entrances to their homes, so that the Angel of Death would skip over their houses. The wicked son, who excludes himself from the community and does not believe in Hashem or His commandments, would not have marked the entrance to his home and so would have fallen prey to the Angel of Death.
ALTER OF NOVARDOK: Had the Jews not trusted Hashem and followed Him into the wilderness, they would have remained in Egypt, enslaved to Pharaoh. From the wicked son's questions, it is clear that he would not have had the necessary faith to embark on the journey through the desert and would have opted to remain behind in Egypt.
RAV REUVEN KATZ: The wicked son has no interest in the redemption of the Jewish people. He would rather be enslaved to the depravity of Egypt. Had he been in Egypt, he would not have left, but would have chosen to adopt the Egyptian culture.
HASHIR V'HASHEVACH: The korban Pesach is a communal offering. The wicked son excludes himself from the community and therefore cannot be part of the communal redemption.
RAV YECHEZKEL LEVENSTEIN: The purpose of the redemption was to transform us into servants of Hashem. If someone rejects this service, as the wicked son does, he proves himself unfit for redemption.