Is one allowed to use the glass cups provided by a hotel (usually in the bathroom) for drinking? Or is there a Kashrus problem?
Assuming the cup is clean, and you're using it for cold stuff, it's okay.
From a shiur I heard (I think an OU workplace kashrus one? Or was it a kashrus Q&A? It was on yutorah): the rule is that clean non-kosher dishes can be used for cold kosher food "once in a while." R' Hershel Shachter's psak is that "once in a while" is once in thirty days. So you could definitely use the glass the first day you check in; beyond that, if there's a reasonable chance that housekeeping brought you a new glass, then you haven't used that glass for thirty days, so drink away!
Sephardic Jews have the rule that glass never absorbs and thus (if clean) can never become a kosher issue at all, which would give you carte blanche here. But Ashkenazic Jews treat non-kosher glassware the same way we'd treat non-kosher metalware (at least before the fact), hence it took a few more steps to permit it, as discussed here.
Other problematic cases would be if you left the drink in the glass for 24 hours, or the drink was super-salty or super-alcoholic (e.g. straight hard liquor), in which case it could become a "pickling solution" and become non-kosher in a matter of minutes; see here for more on this. But if it's your usual water, juice, milk, you're fine.