If someone learns math in order to better understand a gemarah or studies self-help material as a form of mussar in order to be a better person do these types of learning qualify as talmud torah?
Through the ages there have been various banners over the general category of the interplay between 'Torah' and 'secular wisdom/culture'. Rav Hirsch called it Torah im Derekh Eretz and others call it Torah Umaddah. Within these two approaches the ongoing debate is whether Torah and secular studies (or just 'Wisdom') are both useful but mutually exclusive or whether they should be approached in 'synthesis' and whether maddah is a means or and ends.
Your specific examples, I believe, could include many other things such as (e.g.) botany; which can help you understand certain gemaras (sometimes I wonder how one could understand some gemaras without it), or provide you with peace and tranquility in tending your garden, learning to nuture nature, or as tool to educate children.
Norman Lamm in "Torah Umadda" delineates a number of approaches to the topic and also tackles whether Wisdom (in and of itself) can ever be considered talmud Torah. The sections below correspond to chapters from Norman Lamm's Torah Umaddah:
1) Rambam. Being a master of secular wisdom he (according to Lamm) speaks of secular studies and the "obligation to pursue them as an act of mitzvah". Rabbi Kapah (ha-yahas le'limmudei hol be'yahadut, p.68) "if one studies all these disciplines and sciences which for some reason are called 'secular', in order thereby to attain greater understanding and knowledge of G-d - which is the foundation of all wisdom and without which, according to Maimonides, it is impossible to reach such a state of knowledge - such studies are the holy of holies". "Maimonides... holds that Wisdom has talmud Torah value" (p.139 Torah Umaddah)
2) R S.R. Hirsch - Torah im Derekh Eretz. Torah and Wisdom are not at loggerheads. Torah is the everpresent rock whereas Wisdom is changing and developing. Lamm writes interpreting Hirsch: "they can cooperate, even as the limbs of the body cooperate and coordinate but they cannot interact substantively, even as a sane and balanced person does not interact with or talk to himself... the secular studies help us to understand Torah more deeply...". So it appears from this approach that Wisdom may not directly be considered talmud Torah.
3) Rav Kook approach: "the sacred is not antagonistic to science, but first he reminds us that it vitalises all; it is that which gives life to the secular disciplines. Kodesh and hol are functionally and indissolubly related to each other "The sacred must be established on the foundation of the profane... the stronger the secular, the more significant the sacred". So secular knowledge should be of superior quality if the sacred is to benefit."
4) Hechsher Mitzvah. Wisdom is a means not an end. Hence not talmud Torah.
5) Chassidic Approach. One avodah in chassidut is to sanctify the profane. Lamm argues that it could be extrapolated that secular studies, when undertaken properly can be elevated to the level of kodesh. Perhaps not 'talmud Torah' but of great merit.
While this is largely related to bondonk's answer I would like to narrow the focus a little more closely on the question as asked and expand on relevant points.
The study of mathematics to help in understand the Gemara is an excellent example of what Rabbi Norman Lamm shlita refers to as heksher mitzvah, an act of preparation for a mitzvah, discussed in chapter 8 of his work Torah Umadda. As noted there (somewhat tangentially) this is in compliance with the halachah in Orach Chaim 231 that even an act which is "permitted" to be preformed must be done "for the sake of heaven", i.e. to facilitate serving G-d through preforming mitzvos.
The corpus of "other [branches of] Wisdom" or Madda thus helps one better to understand the depths of Torah. It is a heksher mitzvah... (ibid page 156)
Nevertheless Rabbi Lamm does present a model which he suggests would subsume such [otherwise] secular studies under the category of Limud Torah. The model Rabbi Lamm calls "the Rationalist Model" is discussed in chapter 4 and is said to represent the view of the Rambam. The Rambam understands Pardes as including "Physics and Metaphysics" and Pardes is included in the mitzvah of Talmud Torah (1:12).
For Maimonies, however, these are not mystical or kabbalistic passages. Rather, they represent, respectively, physics and metaphysics. Natural sciences and philosophy are thus the contents of Pardes (page 80-81)
This novel approach is not without its difficulties.
First of all, even though the topics the Rambam identifies with Pardes are included in the category of "secular studies" or "maddah" or "chochmah", not all secular studies are Pardes. Some of the most relevant secular studies are in fact excluded from the category of Pardes as prerequisites. We see for example, that Math is not included in Pardes and therefore not "Talmud Torah": "first study Logic, next the various branches of Mathematics in their proper order" (Guide for the Perplexed 1:34, Friedlander). Likewise the moral preparation for the study of Pardes would probably make any musser from self-help books included in the pre-requisites for learning Pardes (ibid). All the more so does this notion fail when it comes topics which aren't even preparation for learning Pardes!
As such all of these subjects must be done with the intent to enable one to preform the mitzvah of Talmud Torah (or other mitzvah) bringing us back to the notion of "heksher mitzvah".
To further illustrate the difficulty of this idea that learning secular studies can be "Talmud Torah" is that the in his commentary on the Mishnah in Shabbos (chapter 13) Rambam prohibits one to study secular wisdom on Shabbos.
In this way we see the dichotomy with the Rambam's position in Teshuvot HaRambam vol. 3 page 57 disappear because most "chochmah" is not and cannot be Talmud Torah even according to the Rambam. And those topics which he did understand as physics and metaphysics are no longer viable scientifically and it is not at all clear that the reasons which the Rambam understood them to constitute Pardes are transferable to contemporary physics etc.
The Mishnah in Avos (last Mishnah in perek gimmel) says (translation follows Tosfos Yom Tov):
R' Elazar Chisma said: [The laws of] bird offerings and openings of Niddahs are [among] the bodies of law; astronomy and geometry are appetizers for wisdom.
Thus we see that math, important as it might be, is not quite on par with actual halachah.
However, this Mishnah is referring to math in and of itself. Certainly if you're learning math in order to learn Gemara it's considered a part of learning the Gemara. Take, for instance, the Sefer HaChinuch (Mitzvah 258, my own translation):
Amongst the laws of the Mitzvah [of not cheating anyone in business] ... And also there are many topics that are clarified in the books of Algebra and Geometry that split between right angles, obtuse angles, and acute angles, and between equilateral triangles, isosceles triangles, and scalene triangles, and between squares, rectangles, parallelograms, and rhombi. There are so many laws of math that even the largest parchment would not be enough to write down all of these laws... And therefore one must be careful when dividing up land.
And remember with all of this the various measurements that Chazal brought down in the areas of mathematics, such as that which they said that for every amah on the side of a square there are 1.4 amos along the diagonal, or that any circle with three tefachim in its circumference has a diameter of one tefach, or that any square is a quarter bigger than a circle [of the same diameter], and other rules similar to these. They were not exact when they made these rules, and thus one should not rely on them when dividing things between people.
One could bring a proof from this Sefer HaChinuch that one who learns these math principles in order to avoid cheating anyone when selling fields is considered to have fulfilled the Mitzvah; likewise, one who learns math in order to learn Gemara should be considered as if he learned Gemara.
Another proof that secular studies can be used to perform mitzvos can be found in the Gemara in Shabbos (75a, still my own translation):
Says R' Shmuel bar Nachmani in the name of R' Yochanan: How do we know that it is a Mitzvah to calculate astronomy? As it says in the passuk (Devarim 4:6): "And you should guard them and do them, for they are your wisdom and knowledge in the eyes of the nations." What wisdom and knowledge do you have that is in the eyes of the nations (that they, too, can understand - Rashi)? It must be that it's discussing astronomy.
Wait a second. Didn't we just say in the Mitzvah in Avos that astronomy is only an appetizer for wisdom?
It must be, then, says the Tosfos Yom Tov, that the Mishnah is discussing learning astronomy for astronomy's sake, and the Gemara in Shabbos is discussing learning astronomy for Mitzvos' sake.
Note that the Gemara proved astronomy is a Mitzvah because it's a wisdom/knowledge that the nations can understand. This is true of any secular study.
All of the previous answers deal with only one of the two aspects of Talmud Torah - knowing. There's another aspect to it as I assert everywhere, and it is "being engaged in God's wisdom" or דבקות (adherence) even without gathering knowledge. (Like we say the Brocho "לעסוק בדברי תורה", not "ללמוד תורה" or "לדעת תורה".)
In this approach, the only prerequisite for any subject to be considered Talmud Torah is the person's intention. You might be shocked, but one who studies quantum mechanics with the sole intention of clinging to God and knowing His paths can say the Brocho "לעסוק בדברי תורה" on that learning.
This is the tradition I've got from R. Moshe Luria Z"l.