The verse in Tehillim (107:23) from which we derive the law explicitly mentions ships:
יוֹרְדֵי הַיָּם בָּאֳנִיּוֹת עֹשֵׂי מְלָאכָה בְּמַיִם רַבִּים
- "They that go down to the sea in ships, that do business in great waters"
From this I would infer that your first case of swimming is clearly fine (though it's worth noting that the continuing verses clearly also indicate salvation from significant danger). Beyond that, if one is traveling by sea and there is a significant danger, one is clearly obligated to say the blessing. Apparently, the applicability of the blessing when there is no actual danger is subject to dispute.
A further application of the divergent customs is the matter of
somebody who crosses a large river or lake, but not an ocean. The Biur
Halachah (s.v. yordei) writes that the halachah depends on the
divergent customs: According to the custom of Sephard, the blessing is
recited because a large river is no less dangerous than an intercity
road. However according to the Ashkenaz custom, it is possible that no
blessing is recited, because the danger in crossing a large river is
less than in crossing the sea.
From the same site:
Shut Kinyan Torah (Vol. 1, no. 16, sec. 3) writes that somebody who sails across the Channel between England and France does not recite the blessing because the journey is short