One way to look at it, is the notion of going above and beyond, transcending the normal human existence to put Hashem and his Torah/mitzvos first.
As you mention, there are numerous examples, but perhaps we can start with one in Pirkei Avos 2:12 -
רַבִּי יוֹסֵי אוֹמֵר, יְהִי מָמוֹן חֲבֵרְךָ חָבִיב עָלֶיךָ כְּשֶׁלָּךְ, וְהַתְקֵן עַצְמְךָ לִלְמֹד תּוֹרָה, שֶׁאֵינָהּ יְרֻשָּׁה לָךְ. וְכָל מַעֲשֶׂיךָ יִהְיוּ לְשֵׁם שָׁמָיִם:
Rabbi Yose said: Let the property of your fellow be as precious unto you as your own; Make yourself fit to study Torah for it will not be yours by inheritance; And let all your actions be for [the sake of] the name of heaven. (sefaria translation)
The Bartenura explains this mishna to mean:
וְכָל מַעֲשֶׂיךָ יִהְיוּ לְשֵׁם שָׁמָיִם. אַף בְּשָׁעָה שֶׁאַתָּה עוֹסֵק בַּאֲכִילָה וּשְׁתִיָּה וּבְדֶרֶךְ אֶרֶץ לֹא תִּתְכַּוֵּן לְהַהֲנוֹת גּוּפְךָ, אֶלָּא שֶׁתִּהְיֶה בָּרִיא לַעֲשׂוֹת רְצוֹן קוֹנְךָ:
"All of your actions should be for the sake of Heaven": even at the time that you are involved in eating and drinking and in the way of the world (derech eretz), do not have the intention to give pleasure to your body, but rather that you should be healthy [in order] to do the will of your Creator. (sefaria translation)
So we see that our lowly physical existence can be elevated if we choose to maximise our time on this earth by choosing to adopt a healthy lifestyle to best serve Hashem. In other words, the aim of this world is to act with pure motives, thereby synthesising our actions above and beyond.
Take Korach for example. He is seen as the archetype whose actions were not for the sake of Heaven (see Pirkei Avos 5:17). His whole approach is regarded as being self-serving. Unlike the type of arguments that raged between Beis Shammai and Beis Hillel, Korach's dispute was rooted by his own need to further his ambition. It wasn't predicated on personal growth and gaining a closer relationship with G-d. As the Bartenura asserts, Korach and his assembly sought honour and power above all else.
So when we speak about acting for the sake of heaven it means acting with no ulterior motives, endeavouring to act in a more spiritually refined way. This does not have to be regarded as something that transcends human capability, rather the aspiration in itself already is a step in the right direction. It means acting with a heightened awareness that our actions carry weight in the upper realm and our existence on this earth is to serve Hashem to the best of our ability.