The preposition עלי is derived from על while preposition אלי is derived from אל. Probably the shortest explanation of the difference can be found in Sifra Metzora 2:5
"ושלח את הצפור החיה… אל פני השדה" – שלא יעמוד ביפו וישלחנה לים, שלא יעמוד בגבת וישלחנה למדבר, שלא יעמוד חוץ לעיר וישחלנה כלפי העיר. "על פני השדה" – הא אם שלחה וחזרה לתוך ידו – מותרת באכילה.
"and he shall send the living bird (over/at the face of the field.") He should not stand in Yaffo and send it to the sea. He should not stand in Gabbath and send it to the desert. He should not stand outside the city and send it towards the city. Over the face of the field: If he sent it and it returned to his hand, he may eat it.
However, it is somewhat more complex. According to the book "Understanding Rashi" by Rabbi Yisrael Isser Zvi Herczeg, we have the following differention between the prepositions.
The Meaning of Preposition על.
Essay 33 in the book on Parshas Mishpatim titled "Too many word in Rashi's heading" in the second half notes that the preposition על has standard meaning on, by means of, upon, with, which refers to physical placing, as in Bamidbar 21:8 - על נס - on the pole.
Sometimes Rashi superfluously may include a preposition into his heading not only to clarify the word, but also an exception in the usage of a preposition.
For example, Shemos 1:16, על האבנים - upon the stools. When translated directly, it may mean that Pharaoh tells Shifra and Puah to observe birth on "two stones", i.e. stading on two stones. Rashi's comment is defining האבנים as a sitting place where a woman gives birth. It changes the meaning of על to at, i.e. Shifra and Puah had to watch down at the place, in the direction, where a woman gives birth, not their own location.
Another example, Shemos 21:19: על משענתו (lit., with that on which he relies) — i.e. his former healthy state and vigour. Regular meaning of משענתו
is his staff, but as Rashi notes, it is not the meaning of the word in the verse. Therefore, the meaning here is by his own strength.
The example in the question is Sh'muel B 15:4. For the word עלי, Rashi does not give a comment. However, we understand from the context that Absalom is talking about showing lavish affection, agreeing with the grievances of the person as if he is about to place the person upon his shoulders and carry on. Additionally, as we can see from the example in Sifra Metzora 2:5, Absalom's usage of עלי indicates that it was not obligotary for a person to stay in the relationship with Absalom.
The Meaning of Preposition אל and Prefix ל.
Essay 34 in the same book, titled "The Nuances of Prepositions", discusses two prepositions אל and ל, both of which have a meaning to, i.e. coming close to something in some sense. Ba'al haTurim to Vayikra 10:6 shows that there is a difference between the two. R. David HaKohen Zupnik (1916 - 2007, an alumnus of the Mir Yeshiva in Europe) in Kuntres Divrei David haKohein shows that Rashi agrees with the comment of Ba'al haTurim.
When the Torah uses אל for to the stress is on destination. When it uses prefix ל the stress is on the point of departure. As Ba'al haTurim explains: the verse Vayikra 10:6
וַיֹּ֣אמֶר מֹשֶׁ֣ה אֶֽל־אַהֲרֹ֡ן וּלְאֶלְעָזָר֩ וּלְאִֽיתָמָ֨ר ׀ בָּנָ֜יו רָֽאשֵׁיכֶ֥ם אַל־תִּפְרָ֣עוּ ׀ וּבִגְדֵיכֶ֤ם לֹֽא־תִפְרֹ֙מוּ֙ וְלֹ֣א תָמֻ֔תוּ וְעַ֥ל כׇּל־הָעֵדָ֖ה יִקְצֹ֑ף וַאֲחֵיכֶם֙ כׇּל־בֵּ֣ית יִשְׂרָאֵ֔ל יִבְכּוּ֙ אֶת־הַשְּׂרֵפָ֔ה אֲשֶׁ֖ר שָׂרַ֥ף יְהֹוָֽה׃
And Moses said to Aaron and to his sons Eleazar and Ithamar, “Do not bare your heads and do not rend your clothes, lest you die and anger strike the whole community. But your kin, all the house of Israel, shall bewail the burning that יהוה has wrought.
Ba'al haTurim: He turned his face to Aharon and spoke facing him (פה אל פה). This is why it is written אל אהרון. However, he did not turn his face at Elazar and Ithamar, and that's why [the Torah] is not using preposition אל for them in this verse.
In other words, Elazar and Ithamar looked at Moshe as Moshe was speaking to Aharon and also to them.
In the suggested case of Ezra 9:4 אלי means "to me" - just the physical location, meaning Ezra, i.e. to join forces with Ezra and not to receive his emotional or any other support by comparison with Sh'muel B 15:4.
