I remember learning (somewhere in talmud but can't remember where) that the kohanim were divided into 24 groups and they rotated service in the temple. And they went to Yerushalayim for the chagim, of course. My question is: what did they do the rest of the time? They didn't own land, so I presume they lived in the Levitical cities. Doing what?
The kohanim were divided into 24 mishmarim, and each mishmar was divided into batei av. So (aside from holidays when all of the kohanim worked), kohanim worked in the beit hamikdash only 2 days a year.
The answer that I heard (and I forget who said it, probably the Chofetz Chaim) was that a single day working in the beit hamkidash took 6 months of preparation. The kohanim were supposed to be free to be torah scholars, and that's why they're supported by terumah.
Of course, there were kohanim who were amei ha'aretz. Since their knowledge of Torah doesn't reflect spending 6 months of study between visits to the beit hamikdash, they must have spent their time running businesses or something like that. (And it should be noted, that the gemara strongly recommeded against giving one's terumah to kohanim who were amei ha'aretz.)