Originally, Yizkor was said on Yom Kippur and only later was it extended to Shelosh Regalim. Why was it not said on Rosh Hashanah which is Yom Hazikkoron? It would seem more appropriate to say it then rather than on Shelosh Regalim, as it is contrary to the spirit of 'veSomachta beChagecha'.
Divrei Torah Volume 3 #61 mentions a reason that I have heard - which he says is inaccurate. The reason he mentions is that Yom Kippur is like the last day of Rosh HaShana and we always say it on the last days of the Yom Tov. He says that is not accurate as they are separate Yomim Tovim. He then says that on Rosh HaShana we are only supposed to think of live things כי בר״ה יחשבו רק מעניני חיים and through that we will be written to life.
The reason I always remember hearing when I was younger (and I apologise for the lack of a source), is that the yizkor of shalosh regalim is connected to the krias hatorah of asser te'asser / kol habechor, where we have the phrase "ish kematnas yado" - each man should give according to his ability. This relates to the practice of committing to donating money to tzedakah as part of yizkor.
Since this is the krias hatorah for the final day of pesach, the second day of shavuos, and shemini atzeres (in chutz la'aretz) the custom arose to extend the recital of yizkor to those days, and not rosh hashanah, where it is not read.