There are only around 20 million Jews out of a population of 7.6 billion. Of those 20 million maybe 10% (2 million) are Torah observant. According to the Torah one of the purposes of the Jewish People is to be a light upon the nations, wouldn't that be more effective if instead of a few million observant Jews there were 500 million? (I'm not looking for practical reasons like thousands of yrs of exile & persecution, Jews don't proselytize, intermarriage, etc.)
Rabbi Jonathan Sacks discusses the matter and points out that the Jews remain small because in order to be and remain Jewish, one must constantly make the conscious decision to follow Hashem and refuse to compromise.
Consistently throughout our history there have been those who have attempted to increase our numbers by moving away from the will of Hashem. We can see this by looking at the secular, non-religious, anti-religious, and heterodox streams of Jews and seeing what has happened to them.
Consider the people who followed Avraham and Sarah, the souls that they made, and who disappeared over the next two generations. Only Yaakov and the 12 shvatim continued. As Rashi says, 80% of the Bnai Yisrael never made it out of Mitzrayim. Rabbi Beryl Wein points out that the Jews at one time comprised 10% of the Roman Empire and equaled the population of China.
While you ask what is the hashkafah reason that we are so few, the main answer seems to be that people cannot bring themselves to struggle and strive to serve Hashem and will rationalize their attempts to move farther away.
Even when we had open miracles, Hashem allowed us the freedom of choice to decide what path we would go on.
Rabbi Jonathan Sacks explains
The period in question was the first-century Roman empire. Jews numbered some 10 percent of the empire, and there were many Romans who admired aspects of their faith and way of life. The pagan deities of the Hellenistic world were losing their appeal and plausibility, and throughout the centers of the Mediterranean, individuals were adopting Jewish practices. Two aspects of Judaism stood in their way: the commandments and circumcision. In the end, Jews chose not to compromise their way of life for the sake of making converts. The Hellenistic people who sympathized with Judaism mostly adopted Pauline Christianity instead. Consistently throughout history, Jews have chosen to be true to themselves and to stay small rather than make concessions for the sake of increasing numbers.
Why have divine providence or human choice, or both, eventuated in the sheer smallness of the Jewish people? Could it be, quite simply, that through the Jewish people G‑d is telling humankind that you do not need to be numerous to be great. Nations are judged not by their size, but by their contribution to the human heritage. Of this, the most compelling proof is that a nation as small as the Jews could produce an ever-renewed flow of prophets, priests, poets, philosophers, sages, saints, halachists, aggadists, codifiers, commentators, rebbes and heads of yeshivot; that they could also yield some of the world’s greatest writers, artists, musicians, filmmakers, academics, intellectuals, doctors, lawyers, businesspeople and technological innovators. Out of all proportion to their numbers, Jews could and can be found working as lawyers fighting injustice, economists fighting poverty, doctors fighting disease, and teachers fighting ignorance.
You do not need numbers to enlarge the spiritual and moral horizons of humankind. You need other things altogether: a sense of the worth and dignity of the individual, of the power of human possibility to transform the world, of the importance of giving everyone the best education they can have, of making each of us feel part of a collective responsibility to ameliorate the human condition, and a willingness to take high ideals and enact them in the real world, unswayed by disappointments and defeats.
Nowhere is this more in evidence today than among the people of Israel in Israel: traduced in the media and pilloried by much of the world, yet still, year after year, producing human miracles in medicine, agriculture, technology and the arts, as if the word “impossible” did not exist in the Hebrew language. When, therefore, we feel fearful and depressed about Israel’s plight, it is worth returning to Moses’ words: “The L‑rd did not set his affection on you and choose you because you were more numerous than other peoples, for you are the fewest of all peoples.”
Small? Yes. Still surrounded, as the Israelites were then, by “nations larger and stronger than you.” But that small people, defying the laws of history, outlived all the world’s great empires, and still has a message of hope for humanity. You don’t have to be large to be great. If you are open to a power greater than yourself, you will become greater than yourself. Israel today still carries that message to the world.
Religion is a personal choice.
The answer to your question is that in today's age, it is more common for a Jewish person to choose a secular lifestyle than a religious one.
Without getting too political, a secular lifestyle is simply easier. Less laws, less time devotion, less focus on the afterlife and the future. A secular lifestyle is all about the present and enjoying the physical world, which is attractive to many people.
Also due to many years of persecution, a lot of Jewish people find that they are persecuted less when they blend in with their non-Jewish neighbors.
I hope this answers your question. I'm not sure exactly what you're trying to ask.
You seem to be asking two questions. Why are so many Jews secular and non-observant and why are we a small nation. It is impossible to answer the second question if we do not employ practical reasons, such exile, persecution, the need not to proselytize, intermarriage, etc. Perhaps one reason could be traced to the fact the conversion as a practice did not exist in ancient Judaism. Other answers may consist that Jew do not believe in hell like the Christians, who, therefore saw a need to convert folk. Obviously, the anti-semitism harmed and killed many Jews throughout history, preventing our growth and development as a nation; and yet we are on top of the world in terms of the entertainment business, art, literature, science, military, etc.
To answer your first question, I think Rabbi Marc Shapiro answered this well when he writes that Jews today are unwilling to accept that the world is 5000yrs old, Noah’s flood, demons, astrology and many other notions that sound unlikely, if not impossible to occur. Since people are more educated about their past, people are unwilling to throw away history, archeologists and scientific finds, etc etc. For example, most people can no longer believe that all of humanity was descended from Noah and animals that inhabit places like Australia and the rain forest could not have traveled there from Turkey, much less survive in a hot desert region prior to the flood. If we do accept these notions without evidence, on blind faith, then we are rejecting history that says the contrary. I agree with the Modern Orthodox position and that of Marc Shapiro, that Jews do not have to take the biblical stories literally. Indeed the Rambam understood the garden of Eden story allegorically and Rabbi Kook said that it makes no difference whether or not such a place ever existed. Those traditionalist and fundamentialist who do take the Bible literally do so primarily because they reject modern science and history.
But Judaism does not teach us to suspend our intellect. True, G-d could have created the world 5000yrs ago or, 30 minutes ago, for that matter, implanting all our memories. And yes, G-d could have held Mount Sinai over the heads of three million Jews, but our commentators never said that we must believe in these things in order to be good Torah-observant Jews.
Judaism survived copernicus (Rabbi Slifkin says that virtually all of the rabbis reject his view at first), evolution, the age of the universe, etc etc. and so Rabbi Shapiro, and myself included, are convinced that Judaism will continue to exist. What “type” of Judaism is disputed, but I like to think, based on the evidence provided, that a form of "rational religion"will prevail.
In short, I think the reason most Jews are secular today is simply because they are not exposed to Rational Judaism. But when they are, as is to be expected, I think the number of religious adherents will increase, or so I hope.