Are you allowed to try on clothes in a store if there's a ch'shash of shatnez?
If you can't, then what can one do to know if something fits before buying it?
If one does not yet know that a garment contains shatnez, then one is allowed to try it on to see if it fits. If one already knows that there is shatnez in the garment, here is a machlokes between the Bais Yosef (Sefardim) and the Rama (Ashkenazim) as to whether or not one is allowed to try it on. The Bais Yosef says it is asur, and the Rama says that it would be mutar.
The advice given is to try on a suit that is the same size and style of the suit that is being bought and use that to determine if it fits.
Yoreh De'ah 301:6
When one goes to a clothing store to buy a new suit, pants or dress is it permitted to try it on before having the article tested for shatnez?
Usually when people shop for clothing the shatnez status of the garment purchased is unknown. In this case it is permitted to try on the clothing in the store or in the privacy of one’s home to see if it fits. Even if it is known that these garments sometimes contain shatnez, it is still permitted to try them on, unless one knows that the particular garment he is trying on is shatnez.
In the less common case when you know the garment you want to try on for size is shatnez the halachic opinions vary. If a garment is known to contain shatnez, it is prohibited for those who follow the opinion of the Beis Yosef,(such as many of those of Sephardic descent), to try on the garment in order to buy it. If, however, one wishes to model the garment for another person, some authorities would permit it. Some would even permit the buyer to try on a shatnez garment, provided that he will not buy the garment he is trying on. Rather, he should try on another garment that is identical in size and style to the one he will buy.
For those who follow the opinion of the Rama, it would be permitted to try on a shatnez garment in the dressing room of the store or in the privacy of one’s home. It would also be permitted to try on a jacket or a coat in the store itself, since he does not benefit from wearing such a garment in such circumstances. Trying on trousers, or any other garment one would be embarrassed to be seen without, would be prohibited in the store outside the privacy of the dressing room.
Some are of the opinion that even those of Ashkenazic descent should not try on a shatnez garment. It would then be permitted only in the manner described above according to the Beis Yosef. The custom, however, is to be lenient like the Rama.