Thank you for your question.
One thing you have to ask yourself is, when you go out wearing your kippah on certain days, do people see you and remember that today's date is associated with the Holocaust, or do they think that you're Jewish and out enjoying the day?
I don't think there's anything religiously inappropriate about wearing your kippah. As DanF pointed out in the comments, the basic design and idea of the kippah is found in many other world religions and cultures besides Judaism. Jewish Law mandates that we should keep our heads covered, but does not specify with what. In Judaism, a kippah has become a very traditional way of covering the head but a baseball cap or other hat works just as well.
Back to my second paragraph, the one about what people think when they see you. If you want to remind people that today's date is significant in regards to the Holocaust, then maybe wearing a kippah isn't the best way of doing that. You might think of other things to do, perhaps in addition to wearing your kippah which would illustrate that better than simply wearing a kippah.
I hope this answer helps.