Presenting a strictly Chareidi point of view on this site is like walking into a minefield, but here goes.
There were definitely religious leaders who were against instituting a special day to commemorate the Holocaust, but not all gave their reasoning.
One reason that was given came from Rabbi Gedalia Schor as quoted in Meged Givos Olam. The author there said other gedolim shared this idea as well.
When Hashem does something wonderful for us it is a kiddush Hashem, a sanctification of His name. Every time we mention it and commemorate it, there is another kiddush Hashem.
The opposite is true as well. When Hashem does (or allows) something bad to happen to us it is a chillul Hashem, a desecration of His great name. Every mention of it and commemoration of it is another chillul Hashem.
A time such as Tisha Bi'Av or another period of public mourning, which was already instituted by earlier authorities, was viewed as an opening to tack this tragedy on to. This was the same approach taken by the Rishonim and Achronim concerning the tragedies in their communities. For instance the destruction to them resulting from the crusades and the Khmelnytsky Uprising, (see the Taz in O'ch 493 #2 about this).
From a statistical point of view, the crusaders wiped out the same percentage of Ashkenaz Jews as the Holocaust did and no new day of mourning was instituted, but rather liturgy was added in to the Tisha Bi'Av service.
[Edit: This was the opinion of Rabbi J.B. Soleveitchik. He had a tradition from his uncle Reb Velvel pointing to wording in the liturgy of Tisha Bi'Av that that day was chosen as the singular day for commemorating national tragedies.]
It's also worth noting that the religious communities commemorate the victims almost every single week! On Shabbos when we say Av HaRachamim, they were the holy communities who's lives were taken, along with so many others. And again on all three festivals and Yom Kippur anyone in shul saying Yizkor will have said a special prayer for all victims of such acts. Any group of Jews who did not go to shul and pray will obviously have felt a commemoration sorely lacking for such a fresh wound, and will have gone looking for their own path.
So it is not the Chareidim who haven't commemorated the victims of the Holocaust, they have many such commemorations. But they have followed an apparent trend of commemorations for the last thousand years which have not singled out specific events. The one day a year where tradition does seem to single out singular events is Tisha Bi'Av, and commemorations were added.