As many of the commentators here already pointed out, gematria plays significant role in various aspects of Jewish thought, especially in kabbalah and hasidism. Of course the idea of interpreting letters as numbers can be found in many cultures (say, Pierre Bezukhov in Leo Tolstoy's "War and Piece" uses gematria to find out his own future). It is also true though, that as a hermeneutic tool it has been much more popular among Jewish scholars comparing to, say, European culture, especially in last 500 years. May be this is so because Jews didn't use much arabic numerals, or because the sanctuary status of Hebrew alphabeth, in which Torah has been given.
Although I need to say that both gematria and "biblical codes", which are often based on it, use mainly arithmetical operations only and do not have much to deal with mathematics in the modern sense. But some truly mathematical concepts also pop up in Oral Torah sometimes! Here is an example which is close to your question:
In Sefer Yetzira (2:4-5) it says:
These twenty-two letters, the foundations, He arranged as on a sphere,
with two hundred and thirty-one modes. If the sphere be rotated
forward, good is implied, if in a retrograde manner evil is
intended...
(עשרים ושתים אותיות יסוד קבועות בגלגל ברל"א שערים וחוזר הגלגל פנים ואחור וזהו סימן לדבר אין בטובה למעלה מענג ואין ברעה למטה מנגע:)
One can ask himself -- where does the number 231 come from? Why 231? Well, the answer is in the next lines:
For He indeed showed the mode of combination of the letters, each with
each, Aleph with all, and all with Aleph. Thus in combining all
together in pairs are produced these two hundred and thirty-one gates
of knowledge. And from Nothingness did He make something, and all
forms of speech and every created thing, and from the empty void He
made the solid earth, and from the non-existent He brought forth Life.
כיצד שקלן והמירן אל"ף עם כלם וכלם עם אל"ף, בי"ת עם כלם וכלם עם בי"ת וחוזרת חלילה) נמצא כל היצור וכל הדבור יוצא בשם אחד:)
Indeed, 231 is the number of possible choices of pairs of two letters from the 22-letter alphabeth. This is a special case of a binomial coefficient, which are widely used in contemporary mathematics. From mathematical point of view it is still some ad hoc computation, but comparatively non-trivial and extremely interesting from historical point of view