According to this article, the earliest mention of the significance of dreidel is in the late 18th century, and the first mention of it having been done in the time of Chanukah is from 1890.
So, there is no evidence that it was done since then, and it is not mentioned by rishonim.
Regarding nes gadol haya sham, this source from 1911 implies it, as does ...
I'd say, "mess away!"
While relatively recent Hassidic sources have ascribed all sorts of significance to the dreidel, if I'm not mistaken the earliest sources simply discuss the practice of gambling on Chanukah. (Chavos Yair, if I'm not mistaken.) Dreidel seems to simply be a form of gambling that rabbis originally tolerated at best, that at some point ...
dinonline has interesting things to say on the topic
Nobody can say with certainty when the custom of playing dreidel on
Chanukah first began. The idea is not found in sources from Talmudic
times or even in the era of Geonim and Rishonim. Its first mention is
by Ashkenazi authorities of the Eighteenth Century (though the custom
might be older).
The ספר התודעה explains that it was a game given to the kids to occupy themselves after Chanuka Candle lighting, the game was then infused with a meaning that would strengthen the children's emunah so the time was wasted. Hence the connection of the letters to Hashem doing a great miracle there.
Weighted dice are unfair because someone may know which way they are weighted and take advantage. No number on a die is better than any other all the time. Which number you want depends on the game and on your position within the game.
Dreidels are different. The game is completely one of chance (there is no real use of strategy and, while the order of ...