Rabbi Hertz among others point out that the sending away of Yishmael and his mother was the official method at that time of disinheriting the child of a concubine. This is similar to the gifts given to the children of Keturah before Avraham died. Note, that this occurred when Yishmael was about 15 or so, while he did maintain ties with Avraham and his family.
Thus, this was a matter of ensuring that Yitzchak would be the only legal heir of Avraham. He could not have just banished Yishmael. As Sarah said
And Sarah said to Abraham, "Drive out this handmaid and her son, for
the son of this handmaid shall not inherit with my son, with Isaac."
And Hashem told Avraham
And God said to Abraham, "Be not displeased concerning the lad and
concerning your handmaid; whatever Sarah tells you, hearken to her
voice, for in Isaac will be called your seed.
This was a legal maneuver only as we see later:
Rashi Vayeira 22:3 shows that 35 years later Yishmael was with him when he took Yizchak to the Akeidah
his two young men: Ishmael and Eliezer, for a person of esteem is not permitted to go out on the road without two men, so that if one
must ease himself and move to a distance, the second one will remain
with him. — [from Pirkei d’Rabbi Eliezer, ch. 31; Gen. Rabbah ad loc.,
Tan. Balak 8]
Chayei Sarah 25:9 shows that Yishmael was at the funeral and acknowledged Yitzchak as the heir.
9 And Isaac and Ishmael his sons buried him in the Cave of Machpelah
in the field of Ephron the son of Zohar the Hittite, which faces
Isaac and Ishmael: (Gen. Rabbah 30:4, 38:12) From here [we may deduce] that Ishmael repented and let Isaac go before him, and that is
the meaning of “a good old age” which is stated regarding Abraham
(above 15:15). - [B.B. 16b]