The following is taken from the encyclopedia אוצר ישראל Vol. 5 by R. Eisenstein:
In the beginning "Jews" were known as עברים - Hebrews, named for אברם העברי - “Avram the Hebrew” (Bereishis 14:13) who came מעבר (from the other side) of the River Euphrates. And so too Yosef was referred to as “a Hebrew youth” (Bereishis 41:12), and Jonah the Prophet said “I am a Hebrew” (Yonah 1:9), and it says “And the Philistines said: Behold the Hebrews are coming out of the holes in which they hid themselves” (Shmuel Ⅰ 14:11).
The sons/tribes of Yaakov/Yisrael and their descendants were called בני ישראל - the Children of Israel. And after the Bnei Ephraim who were the ten tribes separated from the kingdom of the House of David in the days of Rechav’am ben Shlomo, and only the Bnei Yehudah and the Bnei Binyamin remained under his rule, the were called יהודים after the tribe of Yehudah who were the main group, and after the name of the land ארץ יהודה (the Land of Yehudah) in which they dwelt. And just as the offspring of Ephraim were called אפרתי, so too were the offspring of Yehudah called יהודי. And after the exile of the ten tribes only the descendants of Yehudah and Binyamin remained in Eretz Yisrael, and they were called יהודים and their language was called יהודית as in Yeshayahu 36:11. And even after they too were exiled they were thus called.
Mordechai who was from the tribe of Binyamin was also called “איש יהודי” (Esther 2:5). In the writings of Chazal, the name Yehudi denotes not just a person’s Jewish roots, but also a person who only believes in the God of Israel, and R. Yochanan taught: Really, Mordechai came from Binyamin, so why did they call him Yehudi? Because he repudiated idol worship, because anyone who repudiates idol worship is called a Yehudi (Megillah 13a). The same gemara also comments on the posuk “And his wife the Yehudiah gave birth to Yered” (Divrei Hayomim Ⅰ 4:18) - why is she called Yehudiah, surely her name was Bisyah? Because she denied idol worship.
However, in the Gemara and Midrashim Jews are still called by their old name ישראל or בן/בת ישראל, and only if the quote from foreign sources do they mention the name Yehudi. In the Gemara all customs which are to do with modest behavior which the daughters of Israel were accustomed in, even though they are not explicitly written in the Torah, are called דת יהודית (Kesuvos 72a).
The word "Yehudi" was in former times was considered a term of insult when used by non-Jews, and therefore if they did not wish to insult a Jew they would call him a Hebrew or Israelite. Even now the name Zhid/Yid or Jude or Jew is considered an insulting term in many countries, but Jews in the enlightened countries try to make the name Jew a pleasant term, and many of the sages of the Jews use this name in reference to themselves.
Those who write about the history of the Jews use the term "Hebrew" up until the time of Yaakov, and the name "Yisrael" from the time of the twelve tribes until the destruction of the first temple, and from then on they use the name "Jew".