As we see in Leading a zimun
It is considered an honour to lead the zimun; there is a system of
priority as to how to select the leader. It is usual to ask a guest to
lead; when there is no guest present, the wisest diner is prioritised;
it is also appropriate to offer the honour to a Kohen, although the
host is entitled to lead whenever he wishes. As such, when inviting
the others to respond to his call to bentsch, the leader asks
permission of anyone present whom he believes to take halachic
precedence. This is achieved by saying birshut – with the permission
of – then mentioning the host, Kohanim, etc., before proceeding.
When the person leading the zimun begins with ברשות he has asked permission of everyone seated there and part of the zimun. Additionally, it is the host of the meal who determines which of the guests is asks to lead the zimun. Once the host has done so, it is considered that he has asked the person who is most chashuv among that group.