in the shaar bitachon (end of chapter 3) this verse is explained:
WHY THE RIGHTEOUS SOMETIMES SUFFER
Nevertheless, I saw fitting to attempt to clarify this matter that should be to some extent satisfactory .
The possible reasons why a tzadik is prevented from obtaining his livelihood without effort and must instead exert himself for it and be tested by it is as follows.
A previous sin for which it is necessary to pay him for it, as written "the tzadik will pay in the land" (Mishlei 11:31)
In the way of exchanging, to pay him more good in Olam Haba (the afterlife), as written "to benefit you in your end" (Devarim 8:16)
To demonstrate his good bearing and good acceptance of suffering in the service of G-d, so that others will learn from him, as you know from the matter of Iyov.
Due to the wickedness of his generation, G-d tests him with poverty, hardship, or sickness to demonstrate/contrast his piety and service of G-d unlike them, as written "Indeed, he bore our illnesses, and our pains he carried them" (Yeshaya 53:4).
The commentaries there explain:
(Pas Lechem: Here G-d knows that the people of his generation won't learn from him and they will get no benefit from this. Nevertheless, G-d tests him in order to demonstrate his level and worth and contrast it with them. That even though they are in enjoyment and he is in a life of suffering, they are ungrateful towards G-d while he is good with Him and serves Him in truth and wholeheartedly. The intent in this is so that they will acknowledge G-d's justice in the end of days when they will see the exceedingly great reward of the Tzadik. And like our sages said in Pirkei Avot 5:3, "Avraham was tested with ten trials..to show his high esteem."
Marpe Lenefesh: Due to the wickedness of the generation, sometimes G-d sends suffering to a Tzadik to atone for the sins of the generation (to avert disasters or the like), and without a doubt, his reward will not be withheld and the wicked will be paid what they deserve, as Rashi explained on the verse he cited)
Conclusion: we cannot understand G-d's ways but in the end everything will be balanced out.