If one is not Lubavitch or Breslov is it correct to read their seforim?
Yes, at least for some non-Lubavitch, non-Breslov people. The proof is from the myriad great Torah scholars who cannot reasonably be described as Lubavitch or Breslov but who quote their books. As just one example, Mishna B'rura extensively cites the Shulchan Aruch of the first Lubavitcher rebbe.
This is a personal Hashkafah question IMHO.
Certainly most Lubavitchers and Breslevors would emphatically say it is good for everyone to read their seforim ! :)
The best approach is to choose a path in self development within Torah that really speaks to your soul. In other words, it is better for you to learn seforim that inspire you to live as a better Jew.
One popular way is to follow your father's way in life and ask him for a path consistent with your family's Torah growth. Another way (and in addition) is to join a Yeshivah and follow their path. Your Rebbeim there will show you their seforim selection for such things.
If your soul reads different seforim with different approaches, you may get the feeling of contradiction or at least mixed messages about how to serve Hashem. If that is happening, you should probably pick one main path and not look elsewhere; or merely dabble in other seforim.
If however you are like me, and wish to drink in all truth and process all approaches, then read everything. However, you should understand the difference in pathways from let's say Mussar compared to Tanya. Asking an experienced Rebbe who follows the path you wish to explore, would be very helpful.
After reading the above, my answer is obviously that it doesn't depend upon you being a Lubavitcher or Breslevor; but rather, if their sefarim inspire you.
I can't speak for Breslov but chasidi chabad is intended for all Jews to learn. The Tanya is a very important work and can and will help anyone who learns it with their avodas Hashem.