On all Yamim Tovim, we (this is the Ashkenazic formulation) say the following formulation in the middle Bracha of Shemoneh Esrei:
ותתן לנו ה׳ אלקינו באהבה את יום ... הזה ... מקרא קדש זכר ליציאת מצרים
And You, HaShem, our G-d, gave us with love this day of , , a holy calling, a remembrance of leaving Egypt.
By nickname, I refer to Zeman Cheiruseinu for Pesach, Zeman Mattan Toraseinu for Shavuos, etc.
Notice the final line - a remembrance of leaving Egypt. This line remains even on Rosh Hashanah and Yom Kippur.
Now, Pesach is obvious why we have this. Shavuos is the culmination of the process begun on Pesach, and Sukkos/Shemini Atzeres is also related, also dealing with Hashem’s succeeding kindness to us in our trek from Mitzraim to Eretz Yisrael.
Yom Kippur I could even hear someone saying was when we got the second Luchos and is therefore related.
But Rosh Hashanah seems really out of place. Why is Rosh Hashanah a remembrance of leaving Mitzraim? What connection does Rosh Hashanah have?