What is the opinion of Jewish authorities on Magic, particularly black magic? Traditionally Jews have denied that anything happens outside of the will of God, yet Deuteronomy 18:10 states that these practices do happen. What is going on?
This is a matter of debate among the medieval Jewish authorities.
Maimonides believed that magic does not exist, and explained the verses that prohibit witchcraft and the like do so precisely because it is meaningless nonsense.
Maimonides, Laws of Foreign Worship 11:16 (Simon Glazer translation, available at sefaria.org):
All of these things are false and spurious, and it was with such that the ancient idolaters misled the peoples of many lands so that they be following them. And it is unbecoming to Israel who are exceedingly wise to be attracted by these absurdities, nor to even imagine that they are of any consequence, even as it is said: "For there is no enchantment with Jacob, neither is there any divination with Israel" (Num. 23.23); and it is again said: "For these nations that thou art to dispossess, hearken unto soothsayers and unto diviners, but as for thee the Lord thy God hath not suffered thee so to do" (Deut. 18.14). Whosoever believes in these matters, and their like, and suppose that there is wisdom and truth in in them, save that the Torah disallowed them, such are none other save from among the foolish and ignorant and are to be included among women and children whose mind is not sound. But wise and sound-minded people know that all these matters which the Torah disallowed are not matters of wisdom but formless nonsense followed by senseless people for the sake of which they abandoned every path of truth. Even because thereof the Torah, in admonishing against all these vanities, commanded, saying: "Perfect shalt thou be with the Lord thy God" (Ibid. 18.13).
And now, know and understand regarding magic, that the Creator (may He be blessed) created everything from nothing and made the upper realms the guides of what is beneath them; and He placed the power of the earth and all that is in it in the stars and constellations according to their motion and direction, as has been demonstrated in the science of astrology . . . However, it was one of His great wonders that He placed within the upper realms alternate ways and forces by which one might change the governance of the realms beneath them . . . But it is the regular governance of the constellations that the Creator (blessed is He) desires, which He placed in them to begin with, and this would be the opposite. This is the secret of magic and its power, such that the rabbis said regarding magical practices that they “contradict the Council Above”; in other words, they subvert the simple forces of nature, which is a contradiction to the upper realms to some extent. Therefore, it is proper that the Torah prohibit them so that the world will be left to its normal function and its natural state, which is the desire of the Creator . . .
In other words, magic is a way of subverting the normal rules of nature, using an alternative system still (ultimately) created by G-d.
לחכמים ולמכשפים: אחשוב שהחכמים היו האנשים שהיו חכמים בחכמת מצרים וענין החכמה ההיא היתה להמציא חכמת הכישוף ויעשו בה פעולות זרות אין מדרך הטבע שיגיעו ממנו וזה אם כשיאחזו העינים ויביאו לחשוב שיעשו מה שאינם עושים אם בשימציאו התחבולות הטבעיות יתחדשו בהם ענינים זרים ידמו מפעולות הכשפים או בשיעשו אלו הפעולות הזרות על דרך הכשוף אם היה אפשר שיתחדשו בדרך הכשוף כמו אלו הפעולות על דרך האמת והנה לא נודע לנו עד היום מהות זה הכשוף וענינו ולזה לא ראינו לדבר בו והמכשפים היו האנשים שהיו מתעסקים בפעולות הכשוף
To the wise men and to the sorcerers: I think that the "wise men" were the men that were wise in Egyptian wisdom. And the topic of that wisdom was to produce magical theory and perform with it strange actions that would not naturally result from it. And this is either by sleight of hand and bringing [people] to think that they did something that they didn't [actually] do, or by producing strange things that are similar to magical activity [but] via the utilization of natural methods, or by performing these strange actions via magic if it is possible in truth for such actions to be generated via magic. And behold, until today we do not know the nature of this magic and what it is, and therefore we have not seen [fit] to speak about it. And the "sorcerers" were the men that engaged in magical activities.
The question here has two parts: 1) According to Judaism, does black magic exist? 2) How does this fit into the Jewish belief that nothing happens against the will of G-d?
The Talmud (Sanhedrin 67b) makes a statement that clearly addresses these issues in a cryptic way:
אמר רבי יוחנן למה נקרא שמן כשפים שמכחישין פמליא של מעלה:
(דברים ד, לה) אין עוד מלבדו אמר רבי חנינא אפילו לדבר כשפים
Rabbi Yoḥanan says: Why is sorcery called keshafim? Because it is an acronym for: Contradicts the heavenly entourage [shemakhḥishin pamalia shel mala]. Sorcery appears to contradict the laws of nature established by God.
The verse states: “To you it was shown, so that you should know that the Lord is God; there is none else besides Him” (Deuteronomy 4:35). Rabbi Ḥanina says: This is true even with regard to a matter of sorcery.
The Ramchal (Daas Tevunos siman 36) explains:
שאף על פי שלפי סדרי המערכה העליונה שרצה הקב"ה וערך, הנה כשפים מכחישים פמליא של מעלה (סנהדרין שם), הנה כשהוא רוצה - מושל בכוחו ומבטלם, והיו כלא היו, ולא כשחשבו הפתאים שיוכלו להשתמש מכליו עצמם כנגדו ח"ו:
That even though according to the order of the array of the heavens that the Holy One blessed is He wanted and arranged, sorcery contradicts the family of the heavens (Sanhedrin 67b), behold when he wants - he rules with his power and nullifies them, and they are as if they never existed, and not as the naive thought, that they could use His utensils themselves against Him, G-d forbid.
Some Talmudic sages believed that magic works while other Talmudic sages disagreed and opposed it. An example appears in the Babylonian Talmud Sanhedrin 67b, which says on 8:15, ‘Rabbi Eleazer said:
“This [the fact that Pharaoh’s magicians could not magically produce lice, as did Aron] proves that a magician cannot produce a creature less than a barleycorn in size.’
Rav Papa said:
‘By G-d! They [the pseudo-magicians] cannot produce even something as large as a camel, but all they can do is create an illusion that they magically created something.”
My own view is that I strongly agree with Rabbi Papa. Magic does not exist.
For support, I will quote Ibn Ezra (Leviticus, 19:31) who says,
"Those with empty brains say 'were it not that fortune tellers and magicians were true, the Torah would not prohibit them.' But I (Ibn Ezra) say just the opposite of their words, because the Torah doesn't prohibit that which is true, but it prohibits that which is false. And the proof is the prohibition on idols and statues...."
Thus, according to ibn Ezra the Torah prohibits magic and fortune-tellers because they are a fallacy and do not work. According to ibn Ezra, you do not need a Torah verse literally saying "magic is false," the very prohibition indicates that.
There is no basis for magic in Judaism. That said, many ancient Israelites believed that angels and demons exist. However, Maimonides later dismissed the notion of demons and referred to angels as the natural forces. Even the Talmud makes it clear that demons are actually hallucinations. It, therefore follows, that many ancient Israelites wrongly believed in sympathetic magic. And that is worked. But the ancient Israelites held many wrong notions about G-d and the world. They failed to see that the world functions according to the laws of nature.
Jewish sage the ibn Ezra states in his biblical commentary (Leviticus, 19:31) that,
"Those with empty brains say 'were it not that fortune tellers and magicians were true, the Torah would not prohibit them.' But I (Ibn Ezra) say just the opposite of their words, because the Torah doesn't prohibit that which is true, but it prohibits that which is false. And the proof is the prohibition on idols and statues....."
What he is saying is that the Torah does not prohibit that which is true but that which is false (ie magic, divination, witchcraft, fortune telling, and necromancy), and the proof is the prohibition of the worship of idols. Thus, people have a duty, a religious duty to develop their minds to improve themselves and society. Maimonides considers it an obligation, a mitzvah to study science and the laws of nature.
G-d created people with intelligence “in the image of G-d.” Thus, G-d desires that people use the five senses to determine between truth and falsehood. Magic does not exist.
 Although he did believe in the efficacy of astrology