Per {Rabbi Avraham Yosef Shlita](http://www.moreshet.co.il/web/shut/shut2.asp?id=156380Rabbi Avraham Yosef Shlita) if someone started saying Kaddish in error and people already answered the first Amen it should be completed until Chatzi Kaddish.
Piskei Teshuvos 131:4 notes 14 & 15 says as follows.
Shaalos U'Teshuvos Lehoros Noson Volume 6 Siman 7 says that when Kaddish was said in error, Tachanun should be recited immediately and then followed by Chatzi Kaddish again.
Halichos Shlomo Tefila Perek 11 Os 2 says that if the error was noted in the middle of Kriyas HaTorah then after Kriyas HaTorah, Tachanun should be recited immediately. (no mention regarding saying or not saying Chatzi Kaddish again)
Shaalos U'Teshuvos Rivivos Efraim Volume 1 Siman 97 says that in a case of an error, after Tachanun is recited no additional Kaddish is said.
Orchos Rabbenu Volume 3 Page 225 mentions a story that happened by the Chazon Ish where the Chazan said Kaddish in error, and the Chazon Ish said to just skip saying Tachanun.
Naharei Aish Likutei Diburim 69 mentions that the Satmer Rabbi - Rabbi Yoel Teitelbaum Zatzal - would not say Tachanun on a day where there was a Yarzheit and a Seuda was being held in conjunction with that Yahrzheit. However if there was no Seuda being held in conjunction with that Yahrzheit then he would say Tachanun. Once there was a Yahrzheit where no Seuda was being held in conjunction with it and the Chazan said Kaddish, skipping Tachanun. He told everyone to immediately say Tachanun at that point. (no mention regarding saying or not saying Chatzi Kaddish again)