This is a complicated question, and one that there will not be many satisfying Rabbinic sources for. The first part of my answer will be showing why this question is difficult, and the second part will be to offer an answer on the understanding that there is no real true answer to this question and therefore you should evaluate my answer based on your own studies and understanding.
The question of who can follow what p'sak is complicated now, and has always been complicated. The original way halacha was to be followed was that if someone moved into a Jewish community, he was to follow their accepted customs and norms. However, this started becoming less and less the case. A big example of this is the expulsion of Spanish Jews in 1492. The expulsed Sepharadim traveled in such vast numbers that rather than absorbing into their new host communities, they established their own competing synagogues, and even tried to convince their new communities to follow the Sepharadi rulings. A teshuva by Rabbi Yosef Qaro was published to address this issue, as the Jews of Egypt were upset because these new Sepharadim were trying to establish Yosef Qaro's laws as binding, rather than the Rambam's. And as you will see by Yosef Qaro's teshuvah, even in his time there were lots of different opinions on who should follow what and he did his best to respond to them:
Who is he whose heart conspires to approach forcing congregations who practice according to the RaMBáM of blessed memory, to go by any one of the early or latter-day Torah authorities?! ... Is it not a case of a fortiori, that regarding the School of Shammai—that the halakháh does not go according to them—they [the Talmudic Sages] said ‘if [one practices] like the School of Shammai [he may do so, but] according to their leniencies and their stringencies’:
The RaMBáM, is the greatest of all the Toráh authorities, and all the communities of the Land of Israel and the Arab-controlled lands and the West [North Africa] practice according to his word, and accepted him upon themselves as their Chief Rabbi.
Whoever practices according to him [the RaMbáM] with his leniencies and his stringencies, why coerce them to budge from him? And all the more so if also their fathers and forefathers practiced accordingly: for their children are not to turn right or left from the RaMBaM of blessed memory.
And even if communities that practice according to the Rosh or other authorities like him became the majority, they cannot coerce the minority of congregations practicing according to the RaMBaM of blessed memory, to practice like they do. And there is no issue here concerning the prohibition against having two courts in the same city [‘lo tithgodedu’], since every congregation should practice according to its original custom…”
Source: Abqáth Rokhél, simán 32. Translation: r. M.S. Bar Ron
You will even find such confusion and contradictory ideas in modern times. 4 years ago the previous Chief Sephardic Rabbi's son made the ruling that all Ba'al Teshuvoth in Israel should follow Sepharadi p'sak, even if they are of Ashkenazi descent. But, that ruling hasn't exactly made lots of headway amongst Ashkenazi Rabbis, who will still encourage anyone of Ashkenazi descent to follow Ashkenazi halacha.
So after looking at how difficult of a subject this is, with even the highest authorities in halacha trying to respond to it, the only answer i can give is this: One may not pick and choose p'saks based upon convenience or preference, but rather, one can follow any valid Rabbinic P'sak if he has studied the question thoroughly and has determined the P'sak to be truthful. And if someone wants to follow one particular Rabbi, then one has to try and remain consistent in following that Rav in all of their stringencies, and leniencies, and no one has the right to try and persuade them to follow anyone else. No one ever follows exlusively one Rav, even the Yemenites differ from the Rambam in certain areas. But building a strong halakhic framework rooted in a solid foundation is key.
And in some cases, there may be a hybridization of both, as maybe one Rabbi whose decisions you follow did not rule on a certain situation, then you will have to search halachic answers from other Rabbeim until you find one that you find to be the truth. For example, if one wanted to follow the Rambam in all things, they would not have answers for issues regarding electricity, dishwashers, airplanes, modern bathrooms, etc. They would need to fill in the blanks so to speak. Another situation may be that a person has discovered that nearly all the Sephardic poskim allowed the use of electricity on Yom Tov and found this ruling to be truthful, and one might have to look through many teshuvoth of those Rabbis to find other rulings that are applicable, rather than rely on the more recent poskim who have since backtracked on this issue. Indeed similar situations always occurred. Jews would often rely on local Rabbis for decisions, but when a situation was too difficult, or their Rabbis had no answer, or the community simply did not agree with the decisions of their local Rabbis, they would write letters to far off Rabbis for answers. An example of this would be the case of the Buddha's hand, a type of Etrog which looks very different than our own but is the same species as the etrog. The Iraqi community in India did not know whether or not they could be used for Sukkot, so they wrote back to their hakhamim in Iraq to ask for a p'sak, rather than rely on their local Rabbeim. They did not see an issue of "picking and choosing," rather, they saw themselves attempting to find the truthful p'sak.