As @Dov and @N.T. wrote in the comments, a couple of centuries isn't a long enough time to be considered having a lasting name, and there is indeed evidence that the name didn't last: In Divrei Hayamim 1:2:23 it says:
"But Geshur and Aram took from them Havvoth-jair, Kenath and its dependencies, sixty towns. All these were the sons of Machir, the father of Gilead."
While Chavot Yair's name lasted until the time of Ezra and Nechemiah who wrote Divrei Hayamim, Novach reverted back to Kenat.
Furthermore, Kenat is mentioned in Yerushalmi Shvi'it 16a as one of the boundary-towns from the time of Shivat Tzion:
"תחומי א"י כל שהחזיקו עולי בבל...וקנת ורפיח..."
Translation: "The boundaries of Eretz Yisrael - everything held by the people who came from Babylon...and Kenat and Rafiach..."
Finally, Josephus in Wars of the Jews mentions a city called Kanatha, which according to Wikipedia, can be identified with Kenat.
While Chavot Yair, the city compared and contrasted to Novach, on the other hand, is mentioned three more times after the time of Moshe: Shoftim 10:4, Melachim 1:4:13 and Divrei Hayamim 1:2:23, with no name-change.
To conclude, the midrash is indeed correct that Novach's name did not last and at some point its name reverted back to Kenat.