I think that this is due to halachic issues:
I recently heard a shiur from R. Asher Weiss discussing taking medication on Yom Kippur.
His bottom-line position was that taking pills orally is considered shelo k'derech achilah, and the pills themselves are defined as neveilah she'einah re'uyah. We pasken (following Rambam) that both of these categories are assur miderabbanan.
Therefore, oral medication is only permitted for a choleh (she'ein bo sakanah). Presumably, the people taking caffeine suppositories do not fall into the cateory of a choleh, which is why they won't take oral pills.
He didn't discuss suppositories at all, but I imagine that taking them is not considered any form of achilah and therefore it is muttar lechatchilah to take them.
One further complication with taking pills orally (even for a choleh) is if water is needed to help one swallow them, as drinking the water is k'derech achilah (and additionally we hold chatzi shiur assur min hatorah). R. Weiss mentioned that some poskim recommend rendering the water unfit by putting some dish soap into it, to make it shelo k'derech achilah. His personal ruling is to add a significant amount of salt to the water.
Thus, even if the person under discussion does have the status of a choleh, I can understand why they may prefer to take suppositories rather than needing to take a pill with unpleasant-tasting water.