Upon investigation, I found in the sefer Mishpetei HaTorah from Dayan Tzvi Shpitz Chelek 2:26 a similar type case.
He relates that Reuvein found an old copy of the Rambam in genizah. He then sold it to an antique seller for 1000 shekalim. The antique seller then noticed after the sale that there are handwritten notes on the side of the sefer with the signature of the Rama. Now the sefer is worth 50,000 shekalim. The question he was asked was this a mekach taus. Dayan Shpitz answered that antiques, in general, do not have set prices, but fluctuate from time to time and therefore the halachos of onaa do not come into play. However, he does note that if this sefer would sell for 10,000 shekalim minimum in any place, then since it was sold for only 1000 shekalim then the seller can claim onaa.
Then he discusses a different twist on the case. That if the book has other parts to it that are not associated with the book. Like if one found a ksav yad in the book which by itself is worth money and the seller did not know of its existence then the buyer can keep it with no complaints. However, he notes that if the seller got that book from yerusha (inheritance) then the buyer would have to return the item.
(He brings the sources to his psak on the bottom of the pages, it is lengthy. It's on pp. 127-131)