Shmiras Shabbos K’hilchasa (3:35 and note 97): He brings down that it is mutar to remove the stems from tomatoes, plums, apples, pears, peaches, and cherries as long as this is done for immediate use.
He brings down two possible rationales for this.
a) The stem is just like a peel that is preventing you from eating the entire fruit. According to this approach it would have to be done just prior to eating like peeling fruits
b) The stem is not technically mixed with the fruit since it is so clearly distinct as a separate item. They are just coincidentally attached. According to this approach this could even be done for later since you just aren’t doing Borer.
Nevertheless he suggests that L’maseh a person should also utilize the technique of the Chazon Ish of holding the stem stationary and pulling the fruit away from it so that you are definitely considered as taking the good from the bad and thus covering for all possibilities.