How to decide where the development of Torah stops being useful

Question

I've met so many people who have a different opinion on this than the one I was raised with. We all probably get it from our chinuch, but it seems there are quite a lot of opinions. I'll list what I've come across:

1. Any ideas^* outside of Tanach itself are innovations that are subject to being treated as not-necessarily reliable^**.
2. Mishna and Talmud are the last sources of authentic Torah development. Everything that comes after is distracting and unreliable, especially if it is anything other than a commentary or compilation on Tanach or Talmud.
3. The Rishonim were the last reliable transmitters of the Oral Tradition, and/or holy tzaddikim able to innovate authentic Torah ideas. Everything afterwards is rather distracting and unreliable.
4. The classic works of the Acharonim are the last bastions of authentic Torah thought and development/revelation; the last true "fundamentals". Since then, we've had lots of distracting development of ideas that need to be taken with much discretion.
5. The Torah is in permanent development in an unbroken chain of leaders and tzaddikim, and all Torah from such sources is holy, Torah Emet, and good to learn (and describing it as unreliable is basically heretical). The later stuff in some ways is more important because it is the most developed and honing in on an "end goal" of full explanation and revelation of Hashem's Torah.
6. None of the above, they have an itemised list of what is reliable, and what is not, based on other factors like history and politics. Under here would likely fall common opinions like "anything other than halacha is unreliable" and things like litvish vs chassidus, kabbalah vs philosophy etc.

I have come across shomer mitzvot people who hold by each of the above, and everything in between (e.g. "I generally hold by 5, but since [this Rebbe/Tzaddik] nothing new or reliable has come" or "I hold by 3, that these Rishonim are good, but these other Rishonim are unreliable").

Of course, there is likely truth in all of the above, but the way it becomes an ideology is what drives my question. People will argue with eachother that they are learning the "best and most reliable" Torah, and those who learn the other stuff are "learning unreliable, distracting ideas and are getting fundamentals wrong". There are others that are parev and say "I'll stick to mine, and you can stick to yours, whatever works for you". And again, everything in between.

This is such a mess. My question is, do we have any hope of finding a fact based answer to help us with this?

For example:

• A universally acceptable source that says that there will definitely be a decline in Torah development, and the focus should be on prioritising the earlier stuff, and ideally avoiding, or taking with a grain of salt the later stuff.
• A universally acceptable source that says that Torah development is always ongoing and holy and it is important to keep up with it and treat it all as Torah Emet.
• Proof that there is no such source, and therefore it is up to us to try and figure out what Torah development stage is reliable, and to what degree etc. (and that we should trust whoever in where this line is drawn, and not trust the others who say otherwise etc)

It would be best if such proofs are not biased, like "I only learn up to the Rishonim because this Rishon said so" or "I learn right up until Chabad maamarim because the Rebbe said". Proofs from logic or experience should also be best avoided.

---

^{* Keeping "ideas" general, although hashkafa seems to be the biggest source of the differences, so that's the bigger part of what I am looking for.
** What I mean by "reliable" is "untainted with falsehood". Some of the positions above might themselves not define it quite like that, but it will be similar such as "untainted with degredation" or "universally valuable, as opposed to valuable only for certain people, but distracting to everyone else". It is for this reason the title just uses the more wholesale "useful".}

Answer 1

The Rambam's Introduction to the Yad is the long answer to this question and his Introduction to the Mishna is the even longer answer. I'll try to break it down based off of my understanding (also adding outside sources).

1. As long as there was the Great Bet Din, their rule bound everyone (Mamrim 1:4).
2. In the days of the Zuggot, arguments started (see Hagigah 2:2). From that point on we basically followed minhag (your LOR, see Shabbat 130a).
3. Sometimes your LOR was overruled by greater Rabbis and he and you would then follow the greater opinion.
4. Once something became the custom of all of Israel (see What is the source of the expression מנהג ישראל תורה?), then everyone was bound by it. This happened with the Mishna and the Talmud (see Intro to Yad).
5. Since then there hasn't been one corpus accepted by all Jews but there has been some big ones such as Sefardim accepting Maran.
6. Since basically every Jew, Sfardi and Ashkenazi, moved around in the past hundred years, the concept of minhag and LOR has gotten really confusing and its basically each man to his own (see Shoftim 17:6) as long as he knows halacha.
7. In the recent years, minhag has now shifted from Sfardi vs Ashkenazi to Israel vs America. For example using light conditioner before the mikvah in Israel is considered a proper thing (see here and here) and in America is still frowned upon (see here, sorry there is no "real" sources for this one yet. Another one from personal experience is if a students should keep one or two days of Hag in Israel. From personal experience, Israeli rabbis tend to lean toward keep 1 and American lean toward keeping 2.)

Regarding hashkafa, one is allowed to explain the Torah and even the Mishnah (see Tosfot Yom Tov Nazir 5:5) however they want as long as there is no Halachic difference. Many times Hashkafa will cross over into halacha. Prime example is how the wording of prayer changed after the Zohar was revealed and the Arizal's understanding of it.

Answer 2

This Alshich HaKodesh brings support for there being a continual development of Torah, and demonstrates that it is k'dai to keep up.

He brings proof from Midrash Tanchuma, a rather early midrash, which he mentions in his commentary on Mishlei, which he brings up in this exquisite commentary on Shir Hashirim 1:1, that the Torah will be continually developed. Each generation is zoche to chiddushim, which will explain better and reveal more about Torah than before. The proof is based on the fact that Hashem quotes R' Eliezer to Moshe at Har Sinai. See the commentary on Mishlei for much more detail.

This is how Shir Hashirim came about, for example as King Shlomo's "chiddush", which became cannonized in Tanach! Hashem or angels have been singing it in his name since at least Har Sinai!

או יאמר בשום לב אל מאמרי רז"ל יש יאמר כי שיר השירים על הים נאמר מפי מלאכי השרת וי"א בסיני מפיו ית' וי"א באוהל מועד והנה לעומ' הסכמ' רז"ל ע"פ המקרה כי שרת' על שלמה רוח הקודש ואמר שלשה ספרים שיר השירים ומשלי וקהלת ואם אמרנו שנשכחו והחזירו ברוח הקודש איך יאמר אשר לשלמה והלא עתניאל בן קנז החזיר כמה הלכו' שנשתכחו מישראל בפלפולו ולא נקראו על שמו ולמה יגרע על היות מכח פלפול אך לזה נשית לב אל אומר' אשר לשלמה ולא אמר שיר השירים לשלמה כד"א שיר המעלות לשלמה אמנם הוא כענין מאמרנו בביאור ספר משלי על תאמרו יתברך בימי מרע"ה אליעזר בני או' עגלה בת שנתה כו' והלא כמו זר נחשב יחשוב מאמרו ותורתו בשם איש האדמה ולא עוד כ"א כמה דורות טרם הולדו אך הוא כמדובר שם כי כל נפש ונפש מישראל יש לה חלק בתורה מתייחס אל בחינתה וזה חסדו אשר יעשה הוא יתב' עם כל איש ואיש לבלתי חדש בעולם שום דבר מהמתייחס אל בחינת נפשו רק מפיו בכל דור ודור ולא עוד כ"א גם הוא ית' עצמו ולא יזכיר מאמר הקדו' ההוא רק בשם בעליו כי מאז טרם יולד יאמר זה חלק פלוני כן הדבר הזה גם שהוא יתברך או מלאכיו אז שררו השיר הזה מאז בהחיל' הם אמרו שיר השירים אשר לשלמה לו' זה שיר מהשירים אשר לשלמה כלו' אשר לו חלק נפשו מעין שבח מאמרו יתברך אליעזר בני אומר פרה בת שתים כו' כי מאז גלה כי הדין הוא לאליעזר בנו הוא חלק נפשו כן אמר שיר השירים אשר הוא לשלמה שלו חלק נפשו ומעותד ליאמר על פיו בהתחדש בעולם ומעתה נאמר אשר הוא לו:

Relevant to our discussion is what Chazal say (Shir HaShirim Rabba 1:1:12): "There are those who say that Shir HaShirim was said by the splitting of the sea, and those who say that Hashem said it at Har Sinai, and those who say He said it in the Ohel Moed". This contradicts the previous Midrash (ibid 1:1:10), where Chazal state that Ruach Hakodesh rested on Shlomo and he said 3 sefarim: Shir HaShirim, Mishlei and Kohelet. If you say that they once knew it and it was forgotten, and he remembered it with Ruach Hakodesh, how can it be attributed to Shlomo? Antiel Ben Kanaz brought back many [forgotten] halachot through incredible pilpullim and they are not attributed to him! Note that the phrase is "אשר לשלמה", it doesn't say "שיר השירים לשלמה", like we say "שיר המעלות לשלמה". It's like I wrote in my commentary on Mishlei (22:17), I explain a Midrash (Tanchuma) where Hashem is quoted saying "Eliezer my son states that the calf must be a year old...". Isn't that strange? Hashem is expounding His own Torah, basing His own words on the way some mortal will understand them born several generations in the future! As we explained there, in Hashem's kindness, each Jewish soul has a portion in the Torah to which he can relate, and in that area he can make a chiddush to the world, and in each generation chiddushim are conceived by Torah scholars who are able to understand them in a way that was not previously understood and Hashem Himself will not state those chiddushim without mentioning the name of the one who will introduce it to the world, as we see in this case of Rabbi Eliezer where Hashem quoted this halacha in his name way before his time. Similarly, Hashem or the ministering angels may have sung Shir HaShirim many generations before Shlomo, yet still it begins with "שיר השירים אשר לשלמה", i.e. "Shir HaShirim which was apportioned to the soul of Shlomo". In the case of R' Eliezer, the halacha was already in his soul at Sinai, and similarly with Shir HaShirim, it belonged to Shlomo's soul even before he lived and hence it is attributed to him and given his name.

It seems that this isn't just referring to halacha, but any Torah.