It is similar to our blessing of גומל חסדים טובים (bestows beneficient kindnesses) in a multifold form, so is אב הרחמים - a father of multifold mercies. Perhaps, it is connected to the fact that grammatically speaking, both main names of Hashem - E-lohim and A-donai - are plural of the forms (E-loah and A-doni).
The language of father comes from the midrash Bereshis Rabbati Vayishlach 12 on Bereshis 33:3 "and he (Yaakov) himself went on ahead of (his wives and children to greet Eisav)
(my translation may need correction) The verse Tehilim 103:13: Like a father has mercy for his children, (so the LORD has mercy for those who fear Him). Our rabbis said that so taught Yoav to David, as he said that he wondered if really a father is more merciful to his children than their mother. Is not she greatly merciful to them as she troubles herself with pregnancy, birth, and raising them? And he (David) said yes, but did not give a fitting answer. So Yoav went to find out to see whether the saying is true and what it means. He encircled entire land of Israel and found one elderly man among many and who had 12 sons and who could provide for their needs with difficulty with earnings of his entire day. Yoav asked him if he would sell him one of the sons and this way he would provide for his wife and other children. The man scolded him. So Yoav went to his wife and spoke to her at lenghth and at the end he bought a son from her for a 100 gold dinars. In the late evening the elderly man came home to eat and did not find one son. He asked where he was. His wife spoke so and so and in a roundabout way showed him 100 dinars. The man understood what happened, he became very angry and did not sleep entire night. In the morning he rose, took those dinars, he took his weapons in his hands and went to find the big man (gavra) who bought his son. The elderly man did not ask details (of the sale) and did not weigh the money until he found Yoav. So he said to him, "I will not be adding to complete the payment, but if you will not retun my son either I will kill you or you will kill me". Yoav heard the man, he took the money and returned his son. So Yoav said: indeed David said correctly: "Like a father has mercy on his children and not like a mother has mercy on her children. Such is a fact, the mother would sell me her son so that she would not be working as hard to provide for her children, but the father exerted himself in providing for the family and was ready to sacrifice himself to kill for his son. That is what it means "like a father has mercy on his children".