We see in the Torah that Haman (המן) came to Bnai Yisrael every morning in the desert. We also see that Haman stopped coming as soon as they entered Eretz Yisrael. Where was he from the time they entered Eretz Yisrael until he joined Achashveirosh?
As recorded in Shmot 16:32-34,"וַיֹּ֨אמֶר מֹשֶׁ֜ה אֶֽל־אַהֲרֹ֗ן קַ֚ח צִנְצֶ֣נֶת אַחַ֔ת וְתֶן־שָׁ֥מָּה מְלֹֽא־הָעֹ֖מֶר מָ֑ן וְהַנַּ֤ח אֹתוֹ֙ לִפְנֵ֣י יְהוָ֔ה לְמִשְׁמֶ֖רֶת לְדֹרֹתֵיכֶֽם׃
And Moses said to Aaron, “Take a jar, put one omer of Haman in it, and place it before the LORD, to be kept throughout the ages.”
As we can see, a bit of Haman was put in a jar. This jar was later kept in the Ark of the Covenant. When the kellim of the Beit HaMikdash were removed to Babylon and later Shushan, the jar was of course taken with them, hence Haman's arrival at Achaverosh's palace.
The hashpa'ah of the produce of Eretz Yisrael was redirected into the man. When the benei yisrael entered Eretz Yisrael, the hashpa'ah reverted back into the actual produce of Eretz Yisrael.
When the beis ha'mikdash was destroyed, the benei yisrael lost contact with this hashpa'ah because they were overcome by physicality - shamanta avisa kasisah.
Haman (ha'min ha'etz) rejoined the benei yisrael to the hashpa'ah of the trees of Gan Eden by threatening them with total annihilation and thus purifying their physical being.
So between the benei yisrael entering Eretz Yisrael and the time of the Purim miracle, Haman was in the trees. Afterwards he was suspended from them.