Refer to the Kitzur Shulchan Aruch 18:2 which provides the reason that @GershonGold already mentioned that the verse is not recited before the Shemone Esrei of Shacharis and Ma’ariv as we don't want to interrupt between the Bracha of Go’al Yisroel and Shemone Esrei. Conversely, when we recite this posuk at Mussaf, Mincha (and Neilah) it is when there is no issue of Semichus of Geulah L’Tefillah:
Before saying, Tehillos le'eil elyon, you should stand and prepare yourself for the prayer of Shemoneh Esrei [Amidah]. You should rid yourself of phlegm and saliva, and anything that may distract your thoughts; then walk back three steps and say: tehillos le'eil elyon, and continue till [Ga'al Yisrael], then return [to your position] by walking three steps forward in the manner of one approaching a king. You should not make any interruption between Ga'al Yisrael and Shemoneh Esrei, not even [in response] to Kaddish, Kedusha, or Barechu, because [the prayer of] redemption must be joined to the [Amidah] prayer. It is best to be precise in concluding the berachah of Ga'al Yisrael simultaneously with the chazzan, because should you finish first and the chazzan afterwards, there is a question whether to answer Amein to the berachah of the chazzan; but when you conclude the berachah with him, you are certainly not required to answer Amein, because you do not answer Amein to your own berachah. (See Chapter Six, paragraph eleven.) In the ma'ariv service, since the berachah preceding the Shemoneh Esrei [does not] end with Ga'al Yisrael, an interruption is permitted, as it is permitted at any place between one chapter and another. [See Chapter 16 paragraph 2] Before praying Shemoneh Esrei, we say the verse: "Hashem, sefasai tiftach" [My Master open my lips]. This is not [considered] an interruption because it is relevant to the prayer; but the verse: "Ki Sheim Hashem ekra," etc. should not be said [before the Amidah], except at musaf and minchah prayers, before saying, Hashem, sefasai tiftach.
As far as why specifically this verse is chosen. I came across the forum thread of the dafyomi.co.il here which links it to a piece from the Midrash Shocher Tov / Midrash Tehillim 29.
The thread reads as follows:
See the siddur of R'Shabsai Sofer, second volume, page 117, where R' Satz discusses the minhag of saying ki shem before shemona esra. At the bottom of page 117 he brings a Midrash Tehillim which clearly links the two.
The response sheds further light:
Thank you for the very interesting sources, I think you've solved our problem, Rav Leibel. Midrash Tehilim (Socher Tov) 29:2 says the following. First it equates the verse "Havu l'Hashem Kavod v'Oz" that King David said with the verse of "Ki Shem Hashem Ekra" -- in both cases Moshe and King David were calling on others to give honor to Hashem, and to pray to Hashem. The Midrash continues that David tells us, in this Mizmor, how to pray: Havu l'Hashem... i.e., to mention Avos, Gevuros, Kedushas Hashem (as the Gemara learns from this Mizmor in Rosh Hashanah 32a; Rashi Berachos 28b DH Havu).
It seems from this Midrash that the words Ki SHem, and Havu l'Hashem, are a source for the prayer of Shemoneh Esreh that we pray. It is still not clear how this can be seen from the words Ki SHem... . Perhaps the "Ki" means not "when," but "just as." Moshe tells us to praise Hashem in Tefilah by following his lead. (In fact, the verses following the verse Ki Shem, in Haazinu, can be roughly formulating the same pattern of Avos, gevuros, Kedushas Hashem.)
Someone else then points out a Sforno on the verse which helps strengthen the claim just made above:
The SFORNO indeed says that the words "Ki Shem Hashem Ekra" mean, "When I pray" (since the words "Korei Hashem" often refer to prayer, see his proofs).
This may indeed be the source for the Midrash that we quoted in our earlier mailing, which asserts that Moshe was asking the Jews to pray. As I mentioned then, the word "Ki" may mean, "just as I" pray, so too should you.