This question is asked by the Be'er Yosef to Leviticus 23:42-43. Here's a summary of what he writes, with some extra sources.
We find two seemingly contradictory aspects to the mitzvah of sukkah. On the one hand, it connotes exile. Some explain 1 this is the rationale for why the mitzvah of sukkah is juxtaposed to Yom Kippur. Just like Yom Kippur atones for a person’s sins, so does exile. A person should atone for their sins by exiling themself to their sukkah. In fact, if G-d forbid a person was decreed with exile due to their sins, they can “fulfill” this decree by exiling themself to their sukkah. We even see this in the laws of sukkah itself. If a person lives all year in a sukkah, they can’t fulfill their mitzvah by living there during the festival of Sukkos 2. They must leave this structure and enter a different sukkah. We see there is an aspect of exile associated with the mitzvah of sukkah. This idea strengthens the above question. Why is a person exempt from the mitzvah if they are in pain? If anything, staying there would be a greater manifestation of the aspect of exile associated with the mitzvah of sukkah 3.
On the other hand, the mitzvah of sukkah expresses joy and pleasure. The Midrash 4 points out that the Torah mentions no idea of joy with the festival of Pesach, only once mentions it 5 with the festival of Shavuos, yet mentions it three times 6 with the festival of Sukkos 7. This is because we are overjoyed after Yom Kippur that our sins have been atoned. As well, Sukkos coincides with the harvest season, where a person is overjoyed at the bounty they have received from Hashem. Many authorities 8 even hold that the physical pleasure one receives from dwelling in the sukkah is the essence of the mitzvah and not just a tangential aspect. This seemingly contradicts the idea that a sukkah connotes exile, which is associated with pain and discomfort.
However, these two ideas are not contradictory at all. In fact, they are one in the same, with the same intent and manifestation. The Torah wants us to be exiled from our permanent dwelling places and to dwell in the sukkah. Yet we are obligated to experience great joy living in the sukkah, enjoying the physical pleasure it provides. All of this is to remind us of the great chesed that Hashem did for us when He took us out of Egypt. In essence, the Jews were exiled from their then homeland of Egypt. Usually when a nation is exiled from one land to another, their experiences are full of pain and suffering. They are starving, dehydrated, their clothing becomes ragged, they can’t sleep, etc. This wasn’t so when they Jews were taken out of Egypt. Hashem surrounded them with the Clouds of Glory, which protected them from the elements of the wilderness. They were given the mun, the manna from Heaven to eat. They were given water from the well of Miriam. The Clouds of Glory even cleaned their clothing that they were wearing 9. They lacked nothing, as if they were living in paradise and not in the wilderness. To remember all of these miracles, we were commanded to dwell in sukkos. There, we are able to experience the seemingly contradictory ideas of exile and joy.
This also explains why a person who is in pain is exempt from the mitzvah of sukkah. If a person is in pain and stays in the sukkah, they are going against the intention behind the mitzvah. They are also considered an simpleton, since they in fact missed the point behind the festival of Sukkos. They should have been filled with joy, taking great pleasure in the sukkah. Through this they would have remembered all the miracles that Hashem performed for the Jews when they left Egypt. They would have then fulfilled the verse “in order for your generations to know that I placed the Children of Israel in sukkos when I took them out of the land of Egypt” 10.
So we see that because of the unique nature behind the mitzvah of sukkah, that's why it has this special dispensation when there is pain.
1 Maharil Hilchos Sukkos § 2, 4. The source for this is Pesikta D’Rav Kahana pg. 457 (Mendelbaum ed., found in Appendix “Parsha Acheres” s.v. באספך מגרנך ומיקבך), brought in Yalkut Shimoni Emor § 653. See also Zohar Emor pg. 103b
2 See Tosafos to Sukkah 2a s.v. כי עביד ליה, Sukkah 8a with Rashi, Rashi to Sukkah 14a, and Shulchan Aruch Orach Chaim 636:2
3 See Pri Tzaddik Sukkos § 18 for a different explanation
4 Yalkut Shimoni Emor § 654. The source seems to be Da’as Zekeinim to Deuteronomy 16:15
5 Deuteronomy 16:11
6 Leviticus 23:40, Deuteronomy 16:14,15
7 Mishneh Torah Hilchos Lulav 8:12 and Zohar loc. cit. (see the commentary of Mikdash Melech ad. loc.) also express that Sukkos has more joy associated with it than any other festival
8 The gemarra in Nedarim 16b says if a person forbids from themself the pleasure of dwelling in a sukkah (by making a neder), they are still allowed to dwell in their sukkah during Sukkos. The gemarra says this is because of the rule that mitzvos were not given for physical pleasure, so the neder isn’t valid. The Machaneh Ephraim Hilchos Nedarim § 25 asks based on Tosafos in Rosh Hashanah 28a s.v. המודר הנאה (who say even without the mitzvah of sukkah there’s physical pleasure dwelling there), the pleasure would seem to be tangential to the mitzvah. Therefore, the rule that mitzvos were not given for physical pleasure shouldn’t apply. All that says is the fulfillment of a mitzvah isn’t considered physical pleasure or benefit (Ran to Nedarim 15b). Why then is someone who makes this neder permitted to dwell in the sukkah? The Oneg Yom Tov Orach Chaim § 50, Teshuvos Chemdas Shlomo Orach Chaim § 23, and Teshuvos Rabbi Akiva Eiger II § 138 explain that the neder would work only when the pleasure is distinct from the fulfillment of the mitzvah. Not true for the mitzvah of sukkah, where the pleasure of dwelling there is the mitzvah itself. Therefore, it would be permitted to dwell in the sukkah despite the pleasure they receive from it, since the rule mitzvos weren’t given for pleasure would apply. Even though the mitzvah of sukkah would seem to have been given for personal pleasure, perhaps there’s a difference between mitzvos being given for pleasure and mitzvos obligating pleasure
9 Rashi to Deuteronomy 8:7
10 Leviticus 23:43