Is there anything problematic about eating in a sukkah after building it but before Sukkot starts? Or I guess likewise after Sukkot is over?
If it's the day immediately after Sukkos (Shemini Atzeres in Israel, Simchas Torah outside it), then there is indeed a problem with eating in the sukkah, in that it resembles "adding to the mitzvah" (although technically it isn't, because there is no intention to perform the mitzvah). The recommended approach in that case is to disqualify it by removing some of the sechach, or (if that's not possible) to bring in utensils that don't normally belong in a sukkah, such as pots, thus demonstrating that you're treating it as a plain old hut.
This isn't necessary anytime after that, though, because it's obvious that it's not meant as an extension of the mitzvah.
So far, this is talking about eating in the sukkah after Sukkos. I haven't found anything so far about the days before the holiday, but perhaps in that case none of this would be necessary, since it would be obvious that no one starts the Yom Tov that early.