Suppose the Chazan starts saying Kaddish in error instead of saying Tachanun - should the Chazan just stop saying Kaddish? Is there a break off point that up to there you stop saying Kaddish and beyond that you just finish it already? Suppose the Chazan finished Kaddish do you say Tachanun now or do you just skip it at this point? (sources)
Per Rabbi Avraham Yosef Shlita if someone started saying Kaddish in error and people already answered the first Amen it should be completed until Chatzi Kaddish.
Piskei Teshuvos 131:4 notes 14 & 15 says as follows.
Shaalos U'Teshuvos Lehoros Noson Volume 6 Siman 7 says that when Kaddish was said in error, Tachanun should be recited immediately and then followed by Chatzi Kaddish again.
Halichos Shlomo Tefila Perek 11 Os 2 says that if the error was noted in the middle of Kriyas HaTorah then after Kriyas HaTorah, Tachanun should be recited immediately. (no mention regarding saying or not saying Chatzi Kaddish again)
Shaalos U'Teshuvos Rivivos Efraim Volume 1 Siman 97 says that in a case of an error, after Tachanun is recited no additional Kaddish is said.
Orchos Rabbenu Volume 3 Page 225 mentions a story that happened by the Chazon Ish where the Chazan said Kaddish in error, and the Chazon Ish said to just skip saying Tachanun.
Naharei Aish Likutei Diburim 69 mentions that the Satmer Rabbi - Rabbi Yoel Teitelbaum Zatzal - would not say Tachanun on a day where there was a Yarzheit and a Seuda was being held in conjunction with that Yahrzheit. However if there was no Seuda being held in conjunction with that Yahrzheit then he would say Tachanun. Once there was a Yahrzheit where no Seuda was being held in conjunction with it and the Chazan said Kaddish, skipping Tachanun. He told everyone to immediately say Tachanun at that point. (no mention regarding saying or not saying Chatzi Kaddish again)