This is from memory from various meforshim (I do not have the sources here). I wrote this up previously but have not been able to find where I did so.
We have the medrashim that Eisav was already leading a band of men. These men were the "family defense force". That is why the servants of Avimelech were unable to attack Yitzchak and instead resorted to "lawfare". That is suing him and claiming that the water in the wells that he dug belonged to them. This means that they were grown up and were independent already.
This is part of why Yitzchak thought that Eisav was still part of the family and helping to support it. He had the idea that Eisav would deal with the outside world and Yaakov would be in charge of the relationship with Hashem. It would be similar to the Yissachar and Zevulun partnership. See Rav Hirsch commentary on Toldos 27:29 as an example.
While we do have those that maintain the principle of "ain mukdam umeuchar baTorah" (the stories in the Torah are not necessarily in chronological order), we do not resort to this unless it is absolutely necessary. In this case, it does not appear to be necessary.
Also the description of the famine and the leaving, seems to have occurred after the death of Avraham. Since we have no mention of Avraham during the growth of the boys until the sale of the birthright (see the medrash that the lentil soup was for the funeral meal of Avraham), it would appear that the famine did not occur in that time. Otherwise, Avraham and his treaty with Avimelech would have been mentioned.
Also, the fact that Avimelech felt free to exile Yitzchak seems to imply that Avraham was not around.
It seems logical that had the famine occurred during the life of Avraham, he would have gone along with the family.
Sice this famine then occured at least 75+ years after the original treaty with Avimelech, it appears that Avimelech's son (at least) is the one mentioned here. AviMelech appears to be an official title, since only a king can be the father of the next king (given legitimate inheritance).
Note that Rav Hirsch suggests that פי כל as two words would mean "the mouth of everybody" , the commander. It could also mean the chief adviser (my own suggestion).