The Rambam addresses your concern directly, in Hilkhot Birkat Cohanim, chapter 15, law 7:
ואל תתמה ותאמר, ומה תועיל ברכת הדיוט זה--שאין קיבול הברכה תלוי
בכוהנים, אלא בהקדוש ברוך הוא: שנאמר 'ושמו את שמי, על בני ישראל; ואני,
אברכם' --הכוהנים עושים מצוה שנצטוו בה, והקדוש ברוך הוא ברחמיו מברך את
ישראל כחפצו
"Do not be perplexed and say, 'What effectiveness can the blessing of
this ordinary/not-scholarly/not exemplary person have?' For the
receiving of the blessing is not contingent on the Cohanim but on the
Holy-One-Blessed-is-He. As it says 'And they shall put my name upon
the children of Israel; and I will bless them.' The Cohanim do the
commandment they were given and the Holy-One-Blessed-is-He, in his
compassion, blesses Israel, according to his desire."
I.e., the section of the Torah that explains the commandment of Birkat Cohanim concludes (Numbers 6:27): "ושמו את שמי על בני ישראל ואני אברכם" "And they shall put my name upon the children of Israel; and I will bless them." God is the "I" who is speaking; it is always God who blesses. Birkat Cohanim is a mitzvah like any other: the purpose of performing a mitzvah is to perform a mitzvah.