Mishna Ketubot 2:10
אלו נאמנין להעיד בגודלן, מה שראו בקוטנן: נאמן אדם לומר, זה כתב ידו של אבא, וזה כתב ידו של רבי, וזה כתב ידו של אחי.... ושהיה איש פלוני יוצא מבית הספר לטבול לתרומתו...
These are trusted to testify as an adult what they saw as a child: a person is trusted to say this is my father's handwriting, and this is my teacher's handwriting, and this is my brother's handwriting... that so-and-so would leave school to dip [in a Mikvah] to eat Terumah...
It's important to have Levi children going out to wash the Kohanim's hands so that they remember that they are Leviyim. This could be one reason to have multiple people washing the hands of an individual Kohein: a Minyan with many more Levi children than Kohanim (or one where Levi children below bar mitzvah are only allowed to wash assisted by an adult/parent).
Everyone else should stay inside and participate in the prayer service since they aren't needed elsewhere and God isn't taking a break from listening to our prayers (Teshuvot veHanhagot 3:48, who calls the practice of multiple Leviyim washing together "unclear", though he recommends even adult Leviyim occasionally make a point of being the one to leave and go wash so as to retain their public presumption of being a Levi).