It could be that birds are included as gidulei karka. See Maggid Mishnah Shabbos 8:7. (בהמה חיה ועוף קרויים הם גידולי קרקע.)
3 Categories of being grown from the ground:
There are things that are rooted and nourish from the ground like wheat, fruit etc.. see Rambam on Mishnayos (Peah 4:1).
There are mushrooms etc... that don't grow from the ground necessarily (like they can grow on trees or objects) but nourish from the ground. (See Rashi Brachos 40:).
There are things that don't nourish or grow in the ground, but do exist on the ground and eat from those things that grow from the ground. These are not called גידולי מן הקרקע but they are called גידולי הקרקע. The distinction is subtle but they are not called "Grown from the Ground" but rather "Ground growth" - my best attempt to translate the difference. See Rashi (Succah 11b: ד׳׳ה כי at bottom of page). Therefore, although some animals and humans are included in a broader category, they are not necessarily considered equivalent to plants etc...
Some Rishonim argue that even if they themselves might be called from the ground, but in comparison to the other categories might not be included with them. See Tosfos Shabbos 73:.
My assumption is that Fish in general do not live on land. And although they might have a few species that can move on land, presumably those would need to be brought in through a drasha specifically. And are not included.
Perhaps a very useful answer is ברייתא דרבי ישמעאל מידה ו:א that says that the animals that "came from the ground" as in Genesis 1:24. Similarly Man is made from the earth. So any verse that presumably has a connection to their creation from ground might be included in the general category. This is a separate but not mutually exclusive approach, though there might be differences of what is included, I am not sure.