the torah is written in a universal form so that every Jew is able to fulfill it.
Nevertheless, G-d's intent is that those who can do more should strive to. This is known as the urge of the understanding.
The prescriptions in Pirkei Avot are in this category.
Most people are not able to fulfill all the words of pirkei avot, so it is not to be looked as 100% halachically binding. Nevertheless everyone at his level of understanding must try to fulfill whatever he can.
This is explained in the Chovos Halevavos, Gate 3 ch.3:
Third, the intellectual urge cannot include equally all who are under
the obligation of service, because some human beings are of limited
intelligence, while some are superior in understanding. But the urge
of the Torah applies equally to all who have reached the status
subjecting them to this service, even though they vary in their
understanding of it, as we have noted at the close of the first
treatise of this book.
It sometimes also happens that an individual falls short in some
duties and exceeds in others. The intellectual stimulus varies in
different individuals in accordance with their capacity of
recognition. But the urge of the Torah is not subject to variation.
Its form is the same for the child, the youth, one advanced in years
and the old man, the wise and the foolish, even though the resulting
practice varies in different classes of individuals. And so Scripture
says in regard to the all encompassing character of the instruction of
the Torah for all the people, "Gather the people together, men and
women, and children, and the stranger that is within your gate, (that
they may hear and that they may learn and fear the L-ord your G-d . .
.)" (Devarim 31:12). Further, it is said, ". . . you shall read this
law before all Israel in their hearing." (Devarim 31:11).