Is there any source to what my Rabeiim taught me in yeshiva that the Avos knew the Torah by "reverse Engineering the Briyah" because Hashem looked into the Torah to create the world so the Avos looked into the world and "reverse-engineered" the Torah?
Great question and welcome to Mi Yodeya.
Although there is a concept of learning about G-d both by lifting one’s eyes to the Heavens and also by studying one’s anatomy carefully (what you describe as reverse engineering the creation), Jewish tradition and Jewish history teach that the Patriarchs received their knowledge of the Torah via Heavenly intervention.
Like with Adam, special angels were sent to teach them all and they also had passed down within their family what the previous generations received.
This began with Adam and the angel, Raziel who came to teach Adam after the sin of the Tree of knowledge of good and its opposite. This angel came to teach Adam and Chava how to do teshuva.
All this is recorded in the book called Sefer Raziel HaMalach.
In addition to Yaacov Deane's answer, it says in Sefer Harokeyach:
כשבא אברם הוא אברהם כשראה את הפלגה חשב איך היה תחלה טרם כל והוא היה בן מ״ח ועסק ג׳ שנים לבדו עד
שאמר לו הקב״ה עסוק עם שם בן נח בן
חכם ישמח אב שבשמים וחשב אברהם
מיוצרו אשר יצרו ואיך יצר העולם עד שחשב ועשה
ברייתא וקראה ספר יצירה וחקר וראה
והבין וחקק וחצב וצירף ויצר וחשב ועלתה בידו לסוף נגלה עליו אדון כל והושיבו
בחיקו ונשקו על ראשו וקראו אוהבי
When Avram -- he is Avraham -- saw the upheaval in the world, he thought about how it must have been before everything. He was 48-years-old at the time, and he spent three years by himself [thinking etc.], until Hashem said to him, "Get involved with Shem the son of Noach." The son of a wise one gladdens his Father in Heaven. Avraham continued to think about the One Who formed him, and how He formed the world, until he came to write Sefer Yetzirah. He delved, saw, understood, carved, pruned, refined, formed, and thought until he succeeded. Ultimately, the Master of All was revealed to him, and He placed him in his bosom and kissed him on his head and called him, "My beloved"...
Also, fifth Rashi on Bereshit 25:27 about Yaacov Avinu:
ישב אהלים. אָהֳלוֹ שֶׁל שֵׁם וְאָהֳלוֹ שֶׁל עֵבֶר: "Dweller of tents" refer to the tent of Shem and Aver
Shem and Aver were teaching Torah in those times, and Avraham, besides working out a lot of emet for himself, was also a talmid of the school of Shem and Aver, as were his children.