What was so special about Avraham's family that he prefers that Eliezer should search there for a wife for Yitzchok (Bereshis 24:40)?
The commentaries on Chayei Sarah 24:4-6 show why this is so.
4 But you shall go to my land and to my birthplace, and you shall take a wife for my son, for Isaac."
5 And the servant said to him, "Perhaps the woman will not wish to go after me to this land. Shall I return your son to the land from which you came?"
6 And Abraham said to him, "Beware, lest you return my son back there.
Some commentaries point out that if Yitzchak married a woman from the local area or if he goes to Padan Aram that she would continue to be influenced by her family. This would cause their children to be influenced as they grow up and they would therefore not become worthy of Hashem's blessing or to inherit the status of Avraham and Yitzchak. Once the future wife comes and settles in Avraham's household, she would learn the ways of Hashem and become assimilated into his household.
Rav Hirsch points out that the people in Padan Aram, though idol worshipers still maintained morality. As a result, the woman from Padan Aram can learn to worship Hashem and to connect the (correct) morality that she had learned as a child to the worahip of Hashem. Thus, she will be able to raise her children to follow the ways of Avraham and Yitzchak. Indeed, we see that even with this care, Yitzchak and Rivkah still failed with Eisav who followed the ways of his uncle Lavan.
When Abraham rejected the daughters of Canaan, and wished for a girl from Aram for his son, we must remember that in Aram too, the people were idolators. So that his reason must have been, not the idolatry of Canaan but only the moral degeneracy of the land. Idolatry is primarily an error of the mind, an intellectual error, which can be corrected and cured. But moral degeneration gets a grip on the whole nature of a person, on the whole depth of his feelings and thoughts, and there, even an Abraham could not hope to find a modest, morally pure, innocent girl for his son, who would bring with her the nobility of fine feelings and morals, as a pearl for his home.
Actually, as the Malbim points out, Avraham does not ask Eliezer to go to his family.
The original directive was:
כִּי אֶל־אַרְצִי וְאֶל־מוֹלַדְתִּי תֵּלֵךְ וְלָקַחְתָּ אִשָּׁה לִבְנִי לְיִצְחָק
But go to my land, to my birthplace and take a wife from there.
IOW: Get away from the land of Canaan and go to my home town. Nice people there; nicer than here, anyway. Especially since the locals are descendants of the cursed Cana'an son of the cursed Cham ben Noah.
Only later when telling over the events does Eliezer change the story to:
אִם־לֹא אֶל־בֵּית־אָבִי תֵּלֵךְ וְאֶל־מִשְׁפַּחְתִּי וְלָקַחְתָּ אִשָּׁה לִבְנִי
But only go to my father's house and to my family and take from there a wife.
In a nutshell, the Malbim explains thus:
He was supposed to go to Avraham's home town and pick a nice girl. The game plan was to go to the well, find a family so poor that they had to send their little girls to draw water and then woo the family with Avraham's enormous wealth.
That's why the water-drawer was rewarded with an expensive set of jewelry: a golden nose-ring weighing a half-shekel, and two gold bands for her arms, ten shekels in weight
But the plan misfired when Rivka insisted on going to the well that day, instead of the maid. וְהִנֵּה רִבְקָה יֹצֵאת. The word הִנֵּה always denotes a surprise.
Once Eliezer discovers this, he changes the story; I need a family member - and that's why I only gave her the jewelry after asking for her identity. And then he pressures them - no worry, if you don't like the idea I'll go to other branches fo the family; Lot or Yisma'el. (Rashi)