There are several points that can be made about this. First, this emphasizes that we are still in galus. If we were free and bringing the karban Pesach, we could not have said this. Additionally, we are making our guests feel better because they see us using "poor man's bread" and it might appear that we cannot afford better. On the other hand, a poor man at the rich man's table would also feel bad.
"Ha Lachma Anya": Rav Moshe Shternbuch - Now You Invite Guests, After Kiddush?
Rav Moshe Shternbuch answers that the guest are already sitting at our
table but they feel really badly about what is going on. They were
invited for the Seder by a man who is also poor. He sits Seder night
with "poor man's bread" on his table. Chazal instituted Ha Lachma
Anya to allay the fears of our poor guest. We tell him that we are
not poor. The poor bread is simply a remembrance to the days when we
were in Mitzrayim.
We then tell him that we have plenty of food to go around and enough
to feed him as well, without giving up our own rations. Therefore,
"Kol Dichfin Yeisei V'Yeichol" all the hungry guests are welcome to
eat in comfort.
To top it all off we tell him that he should not feel like a pauper at
the rich homeowner's table. "Hashata Avdei", we are all really slaves
in this galus. "L'Shana Haba B'Arya D'Yisroel", next year we should
all rejoice together in Eretz Yisroel
Another point is that in order to show that we have been freed, we invite people because a slave owns nothing and can invite no one without stealing from his master. Thus, even though we might be poor and eating "lechem oni" we are still free and we are eating that which belongs to us and which we obtained honestly.
Understanding ‘Ha Lachma Anya’
The Ritvah in his commentary to the Haggadah suggests that on the
night of the Seder every person must envision as if he or she left
Mitzrayim that night. One must feel as if he was just freed from
slavery. A slave owns nothing. Only a free man can own things. A slave
can never invite someone to his meal. Therefore, before we begin our
first meal as free men, we declare that we would like to invite
whomever would like to come. Even though we know that no one can hear
us, we make the declaration to demonstrate to ourselves that we were
just freed from slavery.
Another point is we are saying that we acknowledge that part of the mitzvah of matzah is to ensure that everyone is able to partake of the matza. We therefore announce that we have invited others to be part of the group and we have done our best to ensure that everyone can eat. If somehow someone cannot, we have been ready to have them join our group.
Alternatively, the Vilna Gaon says that the mitzvah of matzah is
connected to the mitzvah of tzedakah. The Yerushalmi in Bava Basra
says that there is an inyan to give maos chitin before Pesach to
enable people to acquire matzah. Based on this we can suggest that the
reason why we make an open invitation at the onset of the Seder is
because the mitzvah of matzah requires us to ensure that others will
have matzah as well. The passage begins with the words Ha lachma anya,
which means this is the poor man’s bread [that our fathers ate in
Mitzrayim] referring to the matzah. As we mention the matzah, it is
imperative that we invite others to join, as the mitzvah of matzah
While it would have been too late to ask anyone to join us for the korbon Pesach, the "kol dichfin" used to be said just before the shechitah of the korbon Pesach. While we can no longer do that, we are announcing that if the korbon Pesach were instituted we would be shechting for a group and we want people to join us in a group for the zecher lekorbon.
Ha lachma anya continues by stating, “Kol d’tzrich yeisei v’yifsach –
whoever needs should come and eat the korban Pesach. Why is it
necessary to invite people to eat from the korban Pesach?
Perhaps this invitation is not meant for poor people alone, but for
anyone, even someone with the means to eat his own korban. The reason
for this is because there is a dispute in the Mishnah in Pesachim 91a
whether one may shecht a korban Pesach for an individual or if it must
be shechted for a group of people. We pasken that the korban Pesach
may be shechted for an individual. However, the Rambam writes in
Hilchos Korban Pesach (2:2) that ideally one should shecht the korban
Pesach only for a group of people, not for an individual. It is
conceivable that it is for this reason that we declare an open
invitation to join in the eating of the korban Pesach.
Finally, the last statement is summing up everything that was said before with the hope that in the future we will be able to fulfill all the thoughts of what we have just said in the best way, by having a group join with us for the korbon Pesach and we will indeed be free under the rulership of Hashem.