Various Midrashim say that Rivka's family put poison in front of Eliezer, in an attempt to kill him (Bereishis Rabbasi by Rav Moshe HaDarshan to Genesis 24:30 and v. 33; Yalkut Shimoni Chayei Sarah § 109; Midrash Aggadah, Midrash Lekach Tov, Sechel Tov, and Midrash HaGadol ad. loc.).
He noticed this and avoided a sure death (Targum attributed to Yonasan to Genesis 24:33), and he ended up successful in his mission to find a wife for Yitzchak. He brought Rivka back home with him.
I could imagine one reason they wanted to kill him is they simply weren't interested in him taking Rivka with them. I also saw in Yalkut Shimoni loc. cit. that they saw his riches and wanted to kill him, I guess to steal them (although I don't get that, since he gave them to Rivka already...I guess in order to keep Rivka and the riches as well).
I was wondering if any commentary suggests a different reason why they wanted to kill him.