In [Bereshit 22:3, Rashi tells us][1] that when Avraham went to sacrifice Yitzchok, the two lads he took with him were Yishmael and Eliezer.

Wasn't Yishmael already gone from Avraham's house? In [Bereshit 21:21][2] we are told that he now dwelt in the Paran Desert.

If Yishmael was no longer in Avraham's house, how did Avraham take him with him to go sacrifice Yitzchok?


  [1]: http://www.chabad.org/library/bible_cdo/aid/8217/showrashi/true/jewish/Chapter-22.htm#v3
  [2]: http://www.chabad.org/library/bible_cdo/aid/8216/showrashi/true/jewish/Chapter-21.htm#21