In Parshas Ki Sisa by the sin of the golden calf, Moshe says to Hashem "if you don’t forgive the Jews, wipe me out of your Torah" . Rashi then says that Moshe said to Hashem, “if you dont forgive them, they will say I wasn't worthy to get mercy for them.”
How do we understand this in depth? Precisely, if Moshe doesn't get forgiveness for them they will say he’s unworthy of getting them forgiveness? How? What does that really mean? They will say to whom? Why would they say that? What’s Moshe’s worry? Does Moshe have a strong claim to Hashem based on this reason to get forgiveness for Bnei Yisroel? If so how?