Is it possible to say that the Jews were only in Mitsrayim for 210 years because they were worked so hard? Would Hashem really do that if he already told Avraham they would be there for 400 years?
Absolutely. That is indeed one of the well-known answers as to how to reconcile these two facts. (I don't recall the original source for this, but an example is here.)
This didn't contradict Hashem's original promise to Avraham, because the wording of that promise was deliberately left ambiguous. The verse (Gen. 15:13) states:
The last clause, "four hundred years," may refer to its immediate antecedent ("they shall enslave and afflict them"), or to the previous one ("your descendants shall be aliens in a land not theirs"). Hashem could have allowed the first meaning to be the correct one, and in that case the Egyptian exile would indeed have lasted a full four centuries; but instead He kindly moved it up so that the Exodus occurred after only 210 years. At that point it was understood in retrospect that Avraham's descendants (beginning with Yitzchak, who was born 400 years earlier) had indeed been "aliens in a land not theirs" (first Canaan, then Charan, then Canaan again, and finally Egypt) for all that time, and that Hashem's promise was fulfilled in that way.