What is significance the Torah wants us to learn from the seemingly exhaustive list of Esav's tribes? (The entire chapter Bereishis 36, which is 43 pesukim long)
Note: I heard the Gra says it hints to our experiences in exile, but a few examples of evidence would be appreciated in an answer.
This is my own answer. I think it's well supported with sources, but I haven't seen anyone explicitly link them together in this way.
My starting point is the Malbim at the beginning of Divrei Hayamim, who counts 70 groups from Avraham's family (counting the families from Seir Hachori, who merged with Eisav's family) to correspond to the 70 nations from Noach and the 70 descendants, later families, from Yaakov. In each case, the 70 people who started each nation or family are listed in the Torah.
In each case, the 70 people listed are the start of a new reality: the nations of the world, the descendants of Avraham and the beginning of monotheism, and the Jewish people. While I can't give a specific lesson to be learned from each of the names, I suggest that they are listed to stress the importance of the new world that those people started.
There is another group of 70 people mentioned in the Torah, but not listed explicitly: the 70 zekeinim who formed the first Beis Din. Rabbi Dosa ben Harkinas in Rosh Hashanah explains why their names aren't listed:
אמר לו, אם באין אנו לדון אחר בית דינו של רבן גמליאל, צריכין אנו לדון אחר כל בית דין ובית דין שעמד מימות משה ועד עכשיו, שנאמר (שמות כד), ויעל משה ואהרן נדב ואביהוא ושבעים מזקני ישראל. ולמה לא נתפרשו שמותן של זקנים, אלא ללמד, שכל שלשה ושלשה שעמדו בית דין על ישראל, הרי הוא כבית דינו של משה.
[Rabbi Dosa] said to [Rabbi Yehoshua]: "If we're going to question Rabban Gamliel's Beis Din, we also have to question every Beis Din from Moshe's time until now. 'And Moshe, Aharon, Nadav, Avihu, and 70 zekeinim went up [to Har Sinai]' - why aren't the zekeinim's names listed? To teach us that any three who form a Beis Din for the Jews are like Moshe's Beis Din".
If the names had been listed, that would indicate that these 70 zekeinim started a new nation within the Jewish people, just like Avraham started a new nation within the world and Yaakov started one within Avraham's descendants. But because the Torah chose not to tell us their names, they were not qualitatively different than anyone else - and any three Jewish men who form a qualified Beis Din have the same authority as they did.
(See the Bartenura and Tiferes Yisrael on that Mishnah, among others, who explain it differently. I haven't seen my explanation anywhere, but it forms a nice connection to the other groups of 70 people, whose names are listed.)
Also see the Malbim in Yisro at the beginning of the Har Sinai experience. Sefaria has the first part but is missing the next few pesukim. It's a beautiful and long piece, but the gist is that the zekeinim really were supposed to be a new reality within the Jewish people [my addition: if that had happened maybe their names would have been listed]. But when we said כל-הדברים אשר-דבר ה' נעשה we indicated that we all wanted to be like the zekeinim and were elevated to their level.
The Torah mentioned all of בני שעיר because all the names listed are actually places which Esau has in his possession. As is stated in Deut. 2, God prohibited the Israelites to dwell there. Therefore God wrote them here to let the borders of Esau’s land be know.
(כתב-יד תלמידי הגר״א, ד״ה א, א:לח)
See notes here for detailed information.
Rabbi Adin Steinsaltz explains (unattributed) that the digression here to recount Esav's progeny is to highlight the fulfillment of the blessings he received from Yitzchak. All of Esav's many descendants came to rule over specific lands (which in turn portends the intertwining of the descendants of Esav's offspring with our own history).
As for why it appears here, this is akin to the Torah's account of Yishmael's generations following Avraham's death (this perek appears after Yitzchak's death); once the Torah "narrative" is essentially finished with a main actor, there's a resultant summary of the legacy. Esav is not directly mentioned anymore following this episode.
