It seems somewhat widespread that Jews that are scared of dogs think that saying ולכל בני ישראל לא יחרץ כלב לשונו will either protect them or scare the dog away or something like that. Does this have any source in authentic Judaism or is it just made up by a primary school teacher 20 years ago?
Yes. this is something that is said in the gemara in order to protect one from dogs. As an example
The Gemara in Masekhet Berakhot (56b) comments that if a person dreams about a dog, upon awakening he should immediately recite this verse from Sefer Shemot that speaks of the dogs’ silence on the night of Yetzi’at Mitzrayim. This should be done quickly, the Gemara urges, as otherwise a different verse about dogs may come to mind first, namely, Yeshayahu’s description of dogs as “azei nefesh” (“brazen” – Yeshayahu 56:11).
While I do not have the citation for scaring away a dog I have found a reference which assumes the custom.
We continuously recited the pasuk “U’lechol Bnei Yisrael lo yecheratz kelev leshono,” which is traditionally recited to protect us from dogs.
The Dubno Maggid is quoted as including the following in one of his derashos:
“A yeshiva rebbe once decided to take his students on a walk through the forest. ‘Now boys,’ he told them, ‘we’re going to be walking through a forest, and there’s a good chance we might come across wild dogs. Well boys – have no fear! All you have to do if we come across wild dogs is to recite the pasuk, "And to all the Children of Israel no dog whet its tongue", and no harm will become us...
Utilizing the pasuk seemed to be a given by the Dubno Maggid when he made the comment in the 1800s--more than just made up by a Rebbi 20 years ago.