This question notes that in Israel, a restaurant cannot receive a hashgacha if they are open on Shabbos. However, in the U.S. this doesn't seem to be the case. What rules do mashgichim rely on to allow the restaurant to be open on Shabbos (let's assume it's owned and managed by non-Jews)? Can a mashgiach enter the restaurant to check on it on Shabbos itself?
Yes a Mashgiach can enter a restaurant, hotel or factory on Shabbos for the purpose of supervising. It happens all the time. In Israel too.
(Source: Experience and people I know who have done it and do it. I once worked a Pesach Hotel over Yom Tov. The 20 minute Seder was particularly fun /sarc).
A restaurant has some particular complications when it is taking customers that pay, but that has nothing to do with the Kashrus per se. More like issues of Misayeah LeDevar Aveira for the Mashgiach and the Hechsher.
Some Kosher certifications have Mashgichim staying in factories over Shabbos to make the cheese Gevinas Yisroel, but those that do it are relying on the opinions that the Jew doesn't actually have to insert the renet, just be there when it happens.