I sent this question a couple of years ago to the OU. They sent me back:
Thank you for contacting the OU.
This has differing opinions among poskim. Some view it as yotzai min
ha'tamei because the non-kosher animal digests the bean and this
improves it. Others view it as pirsha b'alma [waste matter whose Importance has become Nullified] and permit it.
Ask your local Rov for his guidance as to the Halocho l'ma'aseh
Please do not hesitate to contact us again should you have any further
questions.
I did ask my local Rov (Rabbi Kalman Epstein) for his guidance, and he replied that it was permissible. He explained that it is similar to honey, as mentioned by Isaac Moses in the comments to the question.
I have tried the coffee (I bought an eighth of a pound from Porto Rico Importing) and it was so good that it must be assur. :)
As always ask your LOR, although I can see why that might be uncomfortable.