Pardon that I don't have a Rabbinic source to cite, but:
Shemot 20:3 You shall not have the gods of others in My presence.
This would appear to state that there exist the gods of others. So your question may be answered if by "belief" you mean something like 'an understanding that a thing exists'.
What it is that you 'believe' about those things that others call 'gods', and whether you have thus transgressed, is another matter.
Again, I wish I had better sources for the logic of these statements. Drawing from Ariel's answer to this question, citing Rambam on this exact topic, Halacha 3:
Since he may not know the guidelines with which to evaluate [ideas that will lead him] to the truth in its fullness, he may come to heresy.
I think the Rambam indicates that a person can have the capacity to make the evaluations [of this question] with right-discernment. I do not have more for this forum at this time, except to recommend that the Rambam has given a good guideline.