"I never would have taken the oath if I would have known that I was going to break it and get punished"
I don't think that would be a very good reason to nullify an oath. That is like saying "I never would have taken the oath if I knew I had to keep it". Which is obviously a bad reason. Either because it shows he does not know what an oath is, or nothing new was added.
The real question here is a simple one. The hatara of a chochom is l'mafrei'a. If a person transgresses an oath and subsequently nullifies it (retroactively) does he get punished?
First, we must note, beis din does not punish for a broken neder (the Chinuch explains this is because it can be done without action). So, any punishment would be from Hashem. Since Hashem is the True Judge, we cannot say if there is punishment or not.
It is worthwhile to note that the mefarshim dscuss the case where a husband is meifer his wife's neder without telling her. If she "transgresses" the now-nullfied neder, did she do anything wrong? Perhaps this is the same question.