This is an oversimplification, but the question is also very general. I will preface with another two questions. Why is it that if someone rules according to Beis Shamai, or for that matter any opinion in the gemara that we don't rule with he is chastised, even when being stringent, but when it comes to geonim and rishonim there is an allowance to use other opinions, even to be lenient. The answer to this will set up the next question. And the answer is something called Siyum HaShaas (the end of the Talmudic era).
Question number two what is Siyum Hashaas and how does it work?
For this I will try to phrase my answer based on Rabbi Aryeh Kaplan's guidebook to Jewish thought but I don't remember his exact terminology or sources, so I will paraphrase.
Siyum Hashaas, or for that matter the end of any halachic era is when the entire body of the Jewish community is in agreement on a certain set of laws. End paraphrase. I think he was quoting geonim. So when Jews decided to rule like Beis Hillel, whether by vote or heavenly proclamation, this became the end game for Beis Shamai. Same for any ruling in the gemara which was accepted by all before the times of the geonim. Later when new disagreements arose, there was never a point where the entire Jewish world agreed with any given opinion, and if there was it was final. That means that the onion of let's say a Rashba which is not how we rule is worth more in a halakhic scale than the opinion of Abaye whom we didn't rule with. Nothing personal.
Back to your question, being that we have no definitive path to take, whether to be lenient or stringent, we are left wondering how to properly serve God. As such, many times we are chayash, worried, and try to be stringent. This is a requirement when dealing with rules on a deoraisa level, as we can understand very clearly based on the above, but this is also an indication for mitzvah observance in general, which shouldn't seem so foreign, based again on the above.
Imagine two people telling you where a treasure was buried and you don't know who to listen to. Wouldn't you try to figure out a way to take both into your plans? If however it is clear to you which one of them is giving proper directions, obviously the rules change. That's where poskim, or lets say learned people, play in as a factor and say 'I am ruling with this opinion'. That is fine. It's an educated opinion.