In this related question, it is discussed why we daven in the first place. The question still remains: I could very well ask for what I want, even if it's detrimental to me. For that matter, I could even daven that I be successful in an aveirah. Why should we daven for such things? Why don't we just go along with our day, trying to follow the Torah as best as we can, and trust that Hashem will do what is best for us? From that perspective, wouldn't asking Him for something be a failure in this self-test of emunah?
Sometimes prayer is the prerequisite to us receiving things. In chapter 2, verse 5 of Breishit it says the following: ה וְכֹל | שִׂיחַ הַשָּׂדֶה טֶרֶם יִהְיֶה בָאָרֶץ וְכָל עֵשֶׂב הַשָּׂדֶה טֶרֶם יִצְמָח כִּי לֹא הִמְטִיר יְהֹוָה אֱלֹהִים עַל הָאָרֶץ וְאָדָם אַיִן לַעֲבֹד אֶת הָאֲדָמָה:
G-d did not bring rain until Adam prayed for it. Not because He didn't plan on there being grass on Earth - He created grass already! Rather because he wanted to give Adam an opportunity to connect to his Creator and pray for it before having it. The grass was merely 'waiting' under the Earth's surface for Adam to pray, in order for rain to fall and be able to protrude the Earth. This is according to rashi on the verse.
I've heard the analogy given of a faucet. The water may be in the pipes but until we turn the faucet on the water will not fall. So too, for us to get what is in our pipelines we may need to pray for it. Turning on the faucet in the analogy maps to praying.
There is a mistaken assumption -- that the point of davening is to get things. However, the gemara describes prayer as "avodah shebaleiv -- service which is in the heart" (Ta'anis 2a). We pray in order to serve G-d, not in order to get Him to serve us!
So, why do we pray, and in particular, why is so much of prayer a list of requests, if we aren't praying in order to get things?
I can ask the same of many of the conversations I have with my parents. Why discuss my need for a raise with my dad, or parenting worries with my mother? Because part of having a relationship is leaning on others for emotional support. Even when I have little to know expectation of getting any other help from them.
The point of prayer is to turn to one's Parent in times of trouble. Full stop.
If this indeed changes the one who prays to the extent that G-d then decides this new person no longer requires the challenge in question, great. If could be that the primary purpose of the problem was to motivate this act of turning to Him. As our sages describe Sarah's, Rivqa's and Rachel's infertility being caused by "Hashem desires the prayers of the righteous." (Yevamos 64a) Or perhaps just changing the mix of things that need addressing in one's life, one can trade in this challenge for another, hopefully smaller, one. (See Avrohom Yitzchok's answer.)
But if not, the prayer was still successful as long as it remains a practice in staying connected to the Creator and therefore also to what He Created me for.
I heard that the effect of davenning is to change myself so that I am no longer the same person for whom the (present and future) situation has been arranged. Since I have changed, the original arrangements may no longer be suitable for me.
This source discusses the question of how it is possible to change the mind of G-d and responds
התשובה המקובלת היא שהתפילה איננה משנה את רצון הבורא אלא את האדם המתפלל
The accepted answer is that prayer does not change the will of G-d but rather changes the one who prays.