I've heard many times that Hashem is infinitely powerful. But how do we know this? Perhaps he's just very powerful or even all powerful (possessing all power there is, which is possibly finite). Is there a source for the 'infinite'?
The Rambam says that "all powerful" is actually a description of what G-d isn't. (Guide to the Persplexed, 1:15). And if G-d did have attributes, there would have to be a cause for that composite coming together. Since G-d is the Creator, not a created entity, He must be Purely One, without any real attributes. (ibid book 2, intro. postulate #21.) These "attributes" are really approximate statements based on what Hashem isn't.
(Philosophers call this approach to theology the Via Negativa -- the path of denials. The Rambam also describes a second kind of "attribute of G-d", those that describe how his Action looks to us. "Just as He is called compassionate, so too should you be compassionate." -Hilkhos Dei'os 1:6)
Let me start with an easier example of a negative "attribute": Omnipresence.
Where is "1 + 1 = 2". On the one hand, you can say it's everywhere, because there is no place in the universe where 1 + 1 will equal anything but 2. On the other hand, it's a concept. It doesn't have a volume; it doesn't take up space. One can also say "1 + 1 = 2" is nowhere.
Similarly, location isn't really applicable to Hashem, so one can describe G-d as both everywhere and nowhere, depending on perspective. Although we tend to use "shebashayim -- Who is in heaven" rather than "nowhere" to mean spaceless existence.
We talk about G-d always having Been and always will Be. But again, in reality what we are approximating is the idea that Hashem is "lemaalah min hazman -- beyond time." "When" isn't a meaningful thing to say about G-d.
With those two examples setting a pattern, let's look at Omnipotence.
We aren't so much saying that Hashem has Infinite Power vs A Real Lot of Power. Rather, the expression means that Hashem doesn't even involve the concept of power. Hashem's Will simply becomes reality in a way that doesn't involve power. Therefore He can't have too little power to get anything done.
Which is the same effect to our eyes as if Hashem had infinite power -- anything He Wills to happen will happen. Omnipotence.
Your question is truly philosophical, but in Judaism, those things are taken for granted. I'll explain
my personal outlook it based on Rambam Yesodey Hatora 1 (I made it a numbered list, so you can check if you grasp it before you move on):
The fact that Hashem has infinite power is not empirical, i.g. it is not something that we conclude from our observations and conclusions. It is not because of He "split the sea" or He did miracles.
Hashem's omnipotence is theoretical, i.g. it is accepted de-jure, as an axiom. To understand that you have to understand "how G-d works" in Judaism.
When we think of power, we think of an external power - ability to influence things that are external to the source. E.g. you have the power to lift a stone because that stone is external to you. But, can you lift your blood, your brain, and heart? This is meaningless - you may have control over it but not physical "power"
According to Judaism, G-d is ALL, and there's nothing beyond or beside Him (hard to grasp but try to). Therefore, everything we experience as "exists", including ourselves, is (exists), in fact, within G-d Himself.
As such, we assume that Hashem has control (not power) over the Creation as "parts of Himself" (so to speak) and that control is ultimate and unmeasurable (you can call it infinite, but not meaning vast or huge, it is "all there is").