(Source: this article on Torah.org)
If the one who made the mistake is a Jew: Absolutely yes, you must return it.
If the one who made the mistake is a non-Jew: there's lots of discussion, and it appears it's not so clear. But remember: doing so anyways will create a Kiddush Hashem (and is probably the right thing to do).
To quote a relevant story retold by Rabbi Pinchos Bodner:
Rav Yaakov Kaminetsky zt"l recounted that when he was a Rabbi in Lithuania before World War 2, he was asked a question by a resident of his community. The man had purchased stamps from the local post office, and had received more stamps that he had paid for. For such a poor man, the extra stamps were no trifle matter. Nonetheless, Rav Kaminetsky suspected that perhaps the postal clerk was testing the rabbi. This suspicion was confirmed a short time later when the clerk gave him too much change. The rabbi returned the extra money. Many years after the Nazis came and destroyed the community, the rabbi heard that this clerk had saved many Jews, testifying that he tested everybody to assess their honesty, the only trustworthy people were the Jews!