So it turns out that naturally, it's a bit of a machloket.
Khabshush himself testified that there were groups of pro-Kabbalists and anti-Kabbalists or "Baalei P'shat" way before Halévy ever came to Yemen. As he wrote in his book מסעות חבשוש: חזיון תימן (translated by S. D. Goitein), p. 66:
"ובעת שהיינו שם, הגישו המחמירים שבהם, והם ההולכים אחרי ספרי הקבלה, את תלונותיהם לפני מורי יוסף (הלוי), שידון ביניהם ובין אחיהם בעלי הפשט...המחמירים טענו...כפי שמתבאר מספר הזהר הקדוש שהם אומרים עליו שהוא מזמן קדמונינו...ובעלי הפשט השיבו להם ואמרו: מנהג אבותינו תורה, ולא נשנה ממה שירשנו מקדמונינו..."
Translation:
"And while we were there, the most stringent among them, and those that followed the Kabbalistic works, presented their complaints before Mori Yosef (Halévy), that he should judge between them and their brethren, followers of the p'shat...those that were stringent claimed...as is evident from the holy Zohar which they say...is from the time of our ancient forefathers...and the followers of p'shat replied and said: Our forefathers' tradition is the Torah, and we will not change from what we received from our forefathers..."
Goitein noted in a footnote there that the contrasting views of the different groups came about decades before the rise of the Dor Deah group.
However, Yehudah Nini in his essay "מיוסף הלוי ושליחותו לגילוי כתובות שבאיות ועד מחלוקת "עקשים ודרדעים" בשנת תרע"ד", in the book יהדות תימן, argued that the two factions - pro-Kabbalists and anti-Kabbalists - did not deny one another's right to existence. They simply disagreed on what the proper way to pass halachic rulings was - based on Kabbalah as well as other works, or no Kabbalah. In his essay "פולמוס מעניין ווכוח עקר על חכמת הקבלה בין חכמי תימן בראשית המאה", Michael 14, p. 219, he wrote that Khabshush, Rabbi Kafich and Rabbi Sa'id Arusi wanted to impress Halévy, whom they believed to be an important rabbi from Yerushalayim, so they took him on a night-time tour of the Sana'a shuls. Upon entering one of the shuls and seeing all of the Jews studying the Zohar, Halévy said "ברוך דיין האמת" and proceeded to rant off against the study of Kabbalah. The three guides were greatly embarrassed - here was (supposedly) a rabbi from Yerushalayim, and he was calling Kabbalah nonsense. The three stuck to him and were greatly influenced by him. In a sense, they became the first three Temani maskilim.
Later on Glaser also came to Yemen and was in contact with at least Rabbi Kafich and Khabshush. To Rabbi Kafich he gave various scholarly works, as well as possibly three anti-Kabbalistic works written by some of the Acharonim. The most well-known of three is probably ארי נוהם by Rabbi Yehuda Aryeh de Modena. And with Khabshush, he tutored him on the subject that they had a common interest in: Ancient Yemenite inscriptions.
Rabbi Yosef Kafich, in ספונות ב, p. 281, footnote 219, where he wrote the account of the tour of the shul based on what he heard from his grandfather, seems to agree with Nini: The fact that Halévy seemed like a rabbi from Eretz-Yisrael, coupled with the way he played his derision of Kabbalah was enough to sow doubt in the hearts of these men. He concludes his description and short commentary on this story by noting:
"אפשר לומר שכל אותה מחלוקת שהיתה בתימן באה בסיבתו של הלוי."
Translation: "It may be said that all of that dispute [about Kabbalah] that was in Yemen came because of Halévy."
Moshe Gavra in his book תולדות עם ישראל בגלות תימן, p. 208, backpedals from this a bit. He says that while it's true that the fierce no-Kabbalah stance started with these three and strengthened greatly with the creation of Dor Deah, the movement to minimize the usage of Kabbalah started some decades prior (similar to what Goitein said). Rabbis such as Rabbi David Mashriki pushed for more usage of the Shulchan Aruch and other Acharonim, and less usage of Kabbalistic works. Similarly, Mishael Kaspi in his essay "בתי הכנסת כמקור להבנת המבנה החברתי של הקהילה היהודית בצנעא" in "חברה וקהילה", p. 59, points out that it is unlikely that one statement by Halévy was enough to sow doubt in the hearts of these three. More likely, they were already influenced by ideas and movements that had been gaining popularity in Yemen in the previous years.