In Talmud Yerushalmi Megillah Ms. Leiden (פרק א, משנה ח) there is a discussion of the names of G-d, which includes the the earliest spelling of the letter צ to my knowledge: צד'י בי'ת מצבאות.
Other early sources include manuscripts of Bavi Shabbat 12:3 ("צד'י"), Bereshit Rabbat 1:11 ("צאד'י"), Sefer Yetsirah 52 ("צד'י") and some Masoretic manuscripts. Some later sources (e.g. Pirkei Rabbi Eliezer) have the form צ(א)'ד as in Arabic. This is also the name of the letter in the tradition of Yemeni Jews.
Manuscripts of Alphabet of Ben Sira (and some of the Talmud) are the earliest sources I can find with צדי'ק.
Some manuscripts of the Septuagint---the second-temple-era Greek translation of the Tanach---spell out the letters of the alphabet in Hebrew acrostics in Ps. 119 and in Eicha 1-4. There, we find צ spelled σαδη (sade) or τιαδη (tiade), both pointing to a form without the final kof. See Beiträge zur semitischen Sprachwissenschaft from page 126-9.
Granted, all these manuscripts could have suffered from degrees of corruption. However, given the overwhelming sources, it is almost certain that the final kof is secondary.
In Yiddish, צדי"ק appears to be the norm. For instance, Beinfeld and Bochner's Comprehensive
"Yiddish-English Dictionary" and Weinreich's dictionary transliterate the letter as "tsadek." Harkavy's Yiddish-Hebrew-English has צדי'ק as the main entry for the letter.