I know that if I am going to be traveling during Chol Ha'Moed, I know that I can not halachically take down my Sukkah or the Schach early. Does the same prohibition apply to the decorations?
I don't have time to consult the sources right now, but if memory serves correctly, there is no problem disassembling a sukkah during Chol HaMoed. (There are actually portable sukkahs on the market, which are designed to be taken apart and reassembled during the holiday.) You would only run into problems if you wanted to use the materials for some other purpose; they are considered "set aside" for use of the mitzvah until the end of Yom Tov.
Edit: I now see that there is a disagreement about this issue. Some maintain that the very act of disassembling the sukkah is tantamount to removing the materials from the use for which they were designated. Furthermore, even those who are lenient admit that it is a disgrace for the mitzvah to take apart the sukkah, unless it is going to be reassembled somewhere else. (It's not clear to me whether this second reason would apply to decorations. The first reason would definitely seem applicable.)
That being said, a sukkah which was initially built during Chol HaMoed (e.g., a portable sukkah) is certainly allowed to be taken apart if there is a festival-related need to do so. [Source: Piskei Teshuvos 638:3.]
Having just asked the question to my LOR (a Young Israel Shul) the answer is not only does the decorations need to stay up the whole Chag, but the walls and Schach must also stay up till after the end of the Chag.