The Rema writes (YD 88:2) that two guests eating at the same table, one eating cheese and the other meat, should be careful not to drink from the same cup, because the food gets stuck to the cup.
According to those who hold that one should not mix fish and meat whatsoever, would the same also apply to one person eating fish and meat in the same meal, that he would have to clean his glass between them or use a separate one for meat?
Unlike with meat and milk, one is not prohibited from mixing meat and fish in the same utensil. Many are stringent not to eat them together though.
Regarding eating fish and meat one after the other the Rama (SA YD 116:3) writes
the custom is not to wash one's mouth and hands between, but still one should eat and drink something [in between fish and meat].
One sees therefore that he prescribes drinking and does not mention changing glasses. I presume this is because there is no meaningful adhesion of meat/fish on a glass when you drink, and any residual fat is not meaningful.