I'm not so sure that the premise of the question ("all swimming... was unclothed") is necessarily correct. If you look back a century or so - on the contrary, their idea of "bathing suits" was actually clothing that covered all, or at least most, of the body. That may well have been true in earlier times too.
In halachah, we find a discussion about crossing a river on Yom Kippur (Yoma 77b, and from there in Shulchan Aruch, Orach Chaim 613:5); there are cases when one is allowed to walk through water up to his neck (despite the prohibition against bathing on that day), "provided that he doesn't stick his hands out from under the hem of his garment." If I'm understanding this correctly, a person in this situation on weekdays would typically raise up the bottom of his robe to chest level or so, drape it over his arms, and walk through the water; whereas nowadays we might expect a person in such a situation to take his clothes off and carry them on his head. Granted that this is talking about walking rather than swimming, it still seems to show that it was quite normal to go into the water at least partially clothed.
On the other hand, of course, people commonly bathed in rivers and ponds, and naturally they'd do so naked. Shulchan Aruch Harav (Mahadura Basra 2:4), summarizing information from the Gemara (Shabbos 41b), Magen Avraham (2:4), and Pri Megadim (2:4), says that (a) it's not a problem, any more so than in a bathhouse; (b) one should undress and re-dress as close to the riverbank as possible, so as to minimize unnecessary nudity; (c) a man shouldn't cover his privates when going down into the river, since that makes it seem like he's embarrassed about being circumcised; (d) on the other hand, he should cover them (with his hand, or by crouching) when he comes out of the river, since then he's facing other people.
Based on this, it sounds like skinny-dipping isn't necessarily a problem, provided that standards of tznius are maintained out of the water.