There have been many times where I have seen people pull out their Peyos from behind their ears right before Shemona Esrei. Anyone I have asked said there is a source, but I can not locate it. (Supposedly there is a Yesod Vshoresh HaAvodah that mentions it).
Having been through all of the Zohar several times I can tell you that the zohar hakadosh says that when hair is covering the ears of the person praying- it will block the 'ears of heaven' from hearing the prayer. I also remember that the yesod veshoresh hoavoda quotes this zohar somewhere. (unfortunate I cannot recall the location of either the Zohar or the YVH but I am very certain that I have seen them. I assume that this is the source of the custom that you mention.