Dirshu's Doresh Dovid (Moed p. 271) explains that the esrog is the fruit which reveals our intense desires - which can be used for good or not so much. After clearing our Heavenly accounts through teshuva over the Aseres Yemei Teshuva/Yom Kippur, we are able to fix (on our own level) the mistake of Adam Harishon and take the esrog to harness our profound inner desire to serve only Hashem.
Rabeinu Bechaye says that using the Esrog on sukkos atones for the sin of the Aitz Hadaas which was an esrog. (According to shita it was an Esrog) There's also a medrash that implies Adam did the sin 5 days after yom kippur. Meaning the sin took place on sukkos.
So we use an Esrog to atone for the sin that took place that day.