In a Teshuva of the Maharam MiRottenberg (#19) he writes that the angel in charge of conception chooses souls from a chamber (that needs to be emptied before Moshiach can come) to place in the womb. Sometimes the angel makes a mistake and puts a Jewish soul in a non-Jewish mother, and sometimes the inverse. The Jewish soul goes on to become a convert to Judaism, and the non-Jewish soul goes on to convert out of the Jewish religion.
The Chida in מדבר קדמות speaks about the terminology of גר שנתגייר - a convert who converts (not a non-Jew who converts) and says that this soul was at Mt. Sinai at the giving of the Torah, and it just took a while for it to find its way to the Jewish people.
So the idea isn't precisely that there was necessarily a Jewish ancestor (although that might have been the cause for the situation). I don't remember where, but I saw this connected to the idea that Hashem offered the Torah to all the nations who refused. Each nation as a whole refused, but certain individuals agreed, and their souls were the ones at Mt. Sinai.