See the famous Rashi which brings the Shemos Rabbah in Shemos 1:16:
אם בן הוא וגו'. לֹא הָיָה מַקְפִּיד אֶלָּא עַל הַזְּכָרִים, שֶׁאָמְרוּ לוֹ אִצְטַגְנִינָיו שֶׁעָתִיד לְהִוָּלֵד בֵּן הַמּוֹשִׁיעַ אוֹתָם
'אם בן הוא וגו IF IT BE A SON etc. — He was particular only about the male children because his astrologers had told him that there was to be born to a Hebrew woman a son who would become their deliverer (Exodus Rabbah 1:18).
However, it is worth noting that the astronomers did not have exact certainty. Refer to the Rashi later in Shemos 1:22:
לכל עמו. אַף עֲלֵיהֶם גָּזַר (סוטה י"ב), יוֹם שֶׁנּוֹלַד מֹשֶׁה אָמְרוּ לוֹ אִצְטַגְנִינָיו, הַיּוֹם נוֹלָד מוֹשִׁיעָן, וְאֵין אָנוּ יוֹדְעִים אִם מִמִּצְרַיִם אִם מִיִּשְׂרָאֵל, וְרוֹאִין אָנוּ שֶׁסּוֹפוֹ לִלְקוֹת בַּמַּיִם, לְפִיכָךְ גָּזַר אוֹתוֹ הַיּוֹם אַף עַל הַמִּצְרִיִּים, שֶׁנֶּאֱמַר כָּל הַבֵּן הַיִּלּוֹד, וְלֹא נֶאֱמַר הַיִּלּוֹד לָעִבְרִים; וְהֵם לֹא הָיוּ יוֹדְעִים שֶׁסּוֹפוֹ לִלְקוֹת עַל מֵי מְרִיבָה:
לכל עמו This may be translated AND PHARAOH GAVE COMMAND REGARDING ALL (לכל) HIS PEOPLE — Regarding them, too, he made a decree (Sotah 12a). For on the day when Moses was born his astrologers said to him, “To-day their deliverer has been born, but we know not whether he is born of an Egyptian father or of an Israelite; but we see by our astrological art that he will ultimately suffer misfortune through water”. Pharaoh therefore made a decree that day regarding the Egyptians also, as it is said here, “Every son that is born [ye shall cast into the river]”, and it is not stated “[every son] who is born to the Hebrews”. They (the astrologers), however, were not aware that Moses was ultimately to suffer misfortune through the waters of Meribah and not through the waters of the Nile (Exodus Rabbah 1:18; Sotah 12a; cf. also Rashi on Numbers 20:13).
As far as your point:
It is possible that they just wanted some way to oppress the Israelites and made up this prediction...
It is worth noting that Sifsei Chachomim on Shemos 1:16 who says that the decree came precisely as a result of the astronomers' calculations:
אלא על הזכרים שראו אצטגניניו כו'. וא"ת דילמא כדי שלא יפרו וירבו. וי"ל דא"כ היה לו להמית הנקיבות, שאף אם ימית הזכרים וישארו הנקיבות, אם ישאר זכר אחד יכול לעבר כמה נקיבות. [רא"ם]: ולי נראה, דאי ס"ד משום פריה ורביה לחוד הוא, א"כ למה חידש גזרת המילדות, דטעמא מאי הקפיד על פריה ורביה משום טענת ונלחם בנו, והא כבר נתבטלה בגזרת שרי מסים. אלא שראו וכו'. [נחלת יעקב. ע"ש באריכות]:
Only in regard to the male children because his astrologers...We might suggest that Pharaoh’s decree was to prevent Bnei Yisrael from multiplying [and not because of his astrologers]. However, then Pharaoh should have killed the females. For if Pharaoh kills the males and leaves the females, as long as one male remains he could impregnate many females. (Re”m) But it seems to me [to explain Rashi otherwise]: If Pharaoh’s decree was only to prevent Bnei Yisrael from multiplying, why did he innovate the decree pertaining to the midwives? He was concerned about them multiplying because of “he will join our enemies and fight against us,” and this problem was removed by the decree of [appointing] the conscription officers. Rather, this new decree was “Because his astrologers. . .” (Nachalas Yaakov) See further elaboration there.