If the cup is clean of any milk residue, a person who is fleishig can drink from it—and even hot non-milchig beverages—provided that he is not drinking from it during a fleishig meal.
If the non-milchig beverage now in the (clean from milchig residue) milchig cup is cold, there is no transfer of ta'am at all (provided that the cold beverage isn't sitting in the cup uninterrupted for 24hs). And even if the beverage is hot, in which case there is transfer of milchig ta'am from the cup to the hot beverage, this ta'am is not prohibited for someone who is merely fleishig from before—although it cannot be eaten during the course of a fleishig meal.
I am a little bit confused about your title ("Do non-glass cups/straws have to be kashered after use"). Unless the cup became treif, or if it was used with chometz and you now want to use it for Pesach, or if you need to change its status (although note that for Ashkenazim we generally do not kasher utensils to change them from milchig to fleishig, and vice versa, other than when kashering for Pesach), you do not need to kasher the cup.