Having never seen a man cover his hair with a kippah while swimming I always assumed it was permitted to do so. However I am curious if there is any halachic discussion recorded which explicitly permits a man to swim without a head covering.
Yes there is discussion, and since this is Judaism we're talking about, there is of course disagreement.
The Kof-K has a document (PDF) regarding the wearing of the kippah, which footnotes numerous sources for the following sentence:
Wearing a yarmulka is not necessary when a person is actually in the pool, shower, or mikvah
Birchei Yosef 2:5, Sharei Teshuva 3, Ben Ish Chai ibid, Lev Ha’aretz 2:6:52, Kaf Ha’chaim 2:17, Minchas Yitzchok ibid, Yabea Omer O.C. 6:15:7. Refer to Rivevos Ephraim 4:185, Shila D’kaita page 353 quoting the opinion of Horav Elyashiv Shlita, Teshuvos V’hanhugos 1:12.
It also quotes several sources that are stringent. You may wish to CYLOR.