Rashi and Shemos Rabbah both do not say the frog was any larger than average.
The Gemara Sanhedrin 67b does not make sense to be understood that the first frog covered all of mitzrayim.
(I know that all the coloring pages show that but that's not a real proof.)
If you are giving me a source please use a source from a Rishon or earlier.
Some might want to say that the frog was large because why else would the Mitzriyim hit it; and if many other frogs came from it, then the first one was larger.
A few points:
1) See this question and some of the answers- there's not really a source that the other frogs came from of the first one.
2) One of the more famous writings about why the mitzriyim kept on hitting the frog is from the Steipler here. For some reason it seems that he specifically doesn't mention that the frog was big?
3) Maybe the mitzriyim were hitting the frog because Moshe had said the whole plauge would come from it, so the mitzriyim thought that they would kill the frog and outsmart Hashem (no source).
4) This is just a thought, there is no backing to it. And from the fact that all the commentators that I've seen about this don't mention a big frog seems to indicate that there wasn't one.
Here are some of the places where this is assumed as a fact (For some of them it might be easier to do ctrl f and type "frog" or "big frog"):
From Mi Yodeya: This Question, This Answer, This Answer.
Articles about the Makkos: Aish, Aish, PCCK, Aish, Revach, Kol Torah.
Shiurim: TorahAnytime(0:26), TA(7:56), TA(22:53), TA(9:14),TA(1:30).
I have also seen this in children's Parsha stories/ Midrash Says.
(Interesting side note: When this question was asked to me, I looked around in a few seforim but didn't find anything. But when I was going through the Pesukim I saw something else. (Please note that I do not know of any source for this.) Hashem brought the makkos with a יד חזקה, a strong hand. The gematria of יד is 14. If you start at the letter ג from the word הָאֲגַמִּ֑ים in Parshas Vaeira 8:1 and count fourteen letters, you get the ד of הַֽצְפַרְדְּעִ֖ים. Fourteen letters to the ו of וַיֵּ֤ט. Thirteen letters (I know it's off by one) you get the ל of עַ֖ל. Fourteen letters you get the beginning of the word הַצְּפַרְדֵּ֔עַ. That spells out "גדול הצפרדע," the big frog .)