My reference for this sort of question is a book by R Yaakov Hillel called Faith & Folly. R Hillel is a real mekubal that tries to debunk all sorts of superstitions and sort out those that have a halachic basis from those that are forbidden. His book has haskamot (approbations) from R Elyashiv, R Schach, R Zalman Auerbach, R Kanievski, R Kaduri, R Sternbuch and many others.
On palm reading he writes that this art was known to earlier Sages but disappeared over time. Today it is not authentic and shouldn't be relied on (pp. 75-77)
From the Tannaitic through the Gaonic eras, Sages who knew the Torahs
secrets knew also how to read faces and palms, and they passed the
knowledge down fron one to the other (R Sherira Gaon, R Chai Gaon,
Responsa of the Geonim 122). [...] Like the other secrets of the
Torah, the knowledge of reading faces and palms has been lost. All
that remains is a corrupt remnant (Ramban, Derush Torah Temimah). The
physiognomists and palmists of today have turned a lofty wisom into a
trivial pursuit.
Telling the future by reading palms is a violation of
- the prohibition against divining auspicious times (Lechem Mishneh, Mishneh Torah, Avoda Zara 11:9)
- the commandment to have perfect faith (R Gestetner in approbation to Tamim Tiheyeh)
On your question of "how does it work?", he writes
The roots of souls are in the Divine emanations (sefirot). When a soul
is garbed in a body, the soul reveals its roots and its nature in the
body, particularly in the face. One who knows how to read the face can
then expose the souls flaws so they can be rectified through
repentance and good deeds (Or HaChammah on the Zohar Yitro, s.v.
"v'atah techezeh"). This knowledge was given to the Sages of Israel so
that they would know who was suitable for leadership (Zohar ibid) or
worthy of learning the Torah's secrets.
He has slightly more positive things to say about physiognomy (reading a person's face) saying Moshe possessed this wisdom (but didn't use it), the Arizal had it, used it (he could see on a person's forehead what sins he had committed) but refused to transmit it to his disciple R Chaim Vital, and it got lost.