This was a common early belief, and is associated with Egyptians. It is highly likely that there were many books that the Ramban had access to that supported such a theory, although we have no way of knowing which one(s), since the Ramban doesn't refer to these books anywhere else in his writings.
See this excellent Devar Torah/article (on this week's Parsha) by Professor Alexander Klein where he discusses the link between Tzaraas and pigs in Jewish sources, but he does also mention a couple of non-Jewish sources.
One example of where this can be found is in Plutarch's Isis and Osris (here, p21, fn 35 provides more sources):
In like manner they hold the pig to be an unclean animal, because it is reputed to be most inclined to mate in the waning of the moon, and because the bodies of those who drink its milk break out with leprosy and scabrous itching.
Additional note: Chavel notes that in Rabbeinu Bachya/Bechayay, the quote of Ramban reads Sifrei Harefuos, or medical books:
וראיתי בקצת ספרי הרפואות שחלב החזיר אם ינק היונק ממנו יהיה אותו נער מצורע, וזה לאות שיש בכלן סגולות רעות מאד, עד כאן.
These are quoted by Ramban on a number of occasions (see here for search results), but I don't know what particular books Ramban consulted for medical advice... Chavel makes no comments in those places, either.