In the Haggada we read the following verse (Shemos 1:10 from Chabad.org):
הָבָה נִתְחַכְּמָה לוֹ פֶּן יִרְבֶּה וְהָיָה כִּי תִקְרֶאנָה מִלְחָמָה וְנוֹסַף גַּם הוּא עַל שׂנְאֵינוּ וְנִלְחַם בָּנוּ וְעָלָה מִן הָאָרֶץ׃
Get ready, let us deal shrewdly with them, lest they increase, and a war befall us, and they join our enemies and depart from the land.
From this verse, it seems that the Egyptians biggest concern was that the Jews might leave Egypt. This was even more disturbing to them than their own nation being conquered! I mean, the Jews hadn't even been enslaved yet, why did they care so much about losing them?