In Breishit 21:23-24, Avimelech asks Avrahm to swear that his future generations will not destroy his sons or grandchildren.
Bechor Shor commentary says:
תעשה עמדי. כשתבא לידך שלא תגרש גם אתה את זרעי:
When you posses (the land, swear) that that you will not exile my offspring.
If Avraham made this promise, how was King David, or anyone else allowed to destroy / exile the Phillistines?