Some Jews have a practice of keeping kosher at home but eating at non-kosher restaurants, where they will be sure to avoid ordering meat or shellfish. Is there any "point" to this from a halachic standpoint? Since their food is cooked in non-kosher pots and ovens, isn't it just as non-kosher as if they would order meat? Or are they committing less transgressions (and/or less severe ones) through this practice?
They are committing fewer transgressions though I believe it's controversial whether or not the ones that remain are more/less severe since there are texts that indicate that rabbinic prohibitions are more severe (plus there may well be biblical prohibitions still present since shellfish and non-kosher meat are not the only biblical prohibitions). If we assume that less prohibitions are better, that would be an advantage. (There are some counterpoints as well though. For example, often a stringency in one place leads to leniencies in others - "I was strict on kosher so now I can be lax on..." - so the psychology of the individual case would be relevant.)
What to do if you are stuck and there is nowhere other to eat than such a restaurant?
In such a situation should you avoid the non-kosher animals and eat just vegetables?
Well there's a biblical precedent: Daniel found himself in such a predicament in the "palace" of Nebuchadnezzar. He requested to the steward to only serve him from the vegetables and not to serve him the wine but just water to drink
נַס-נָא אֶת-עֲבָדֶיךָ, יָמִים עֲשָׂרָה; וְיִתְּנוּ-לָנוּ מִן-הַזֵּרֹעִים, וְנֹאכְלָה--וּמַיִם וְנִשְׁתֶּה
So obviously it is better, if you're stuck.
So what if you're not stuck but just choose to eat in such a restaurant? Please note that my answer is not giving any licence to go ahead and do so. It is purely philosophical halachic discussion for the sake of looking at the issues:
Some aveirot are worse than others, and that goes by the seriousness of the punishment.
The normal penalty for eating non-kosher is lashes, unless the food contains forbidden blood or fat in which case it is kares, and then you'd have to eat at least an olive-sized portion of such.
Clearly something cooked simply in a non-kosher pot is not going to contain an olive-sized portion of blood or fat so you're probably safe from that and any absorbed non-kosher food may be less than 1/60th too so biblically at least may be "exempt". (This assumes the non-kosher restaurant does not intentionally put these items in and presumably if they wash their pots well anything that "falls" in is accidental)
(Other 2 eating prohibitions that are kares are time-based. Chametz on Pesach and anything on Yom Kippur. Presumably the person would not be eating in such a restaurant on those days).
With vegetables there may still be an issue with bugs, of course. If you "mash" them it's actually less of an issur as you won't be eating one intact. (The G'mara of Makot, which deals with penalties, actually lists the numbers of prohibitions you commit by eating bugs, and eating a whole one makes it worse).
And at worst it would be "shogeg" (careless, rather than intentional) but still should be avoided. (We don't know how exactly this applied in Daniel's day).