The garment mentioned in each case was actually a sign as to who it belonged to. Rav Hirsch points out in Vayeishev 37:3
And Israel loved Joseph more than all his sons, because he was a son
of his old age; and he made him a fine woolen coat.
Rav Hirsch points out the special garment was made to show that Yaakov recognized the special character of Yosef.
פסים, from פסס, to cease to end, seems to be the trimming of the edges
of a garment, which used to be made on all cloaks to mark the
importance of the wearer.
Thus each cloak was identified as the property of the individual and showed who he was and what his position was. Since the cloak was identifiable as belonging to Yosef, then the brothers had to use it to make it appear that a wild animal had torn it from him and gotten his blood on it.
Since the cloak being worn by Yehuda was in a similar manner identified as his, he used it as security so that Tamar could give it back when she was paid. This would be like someone using a check or credit card in modern times.
Again, the cloak that the wife of Potiphar grabbed was also identifiable as Yosef's since he was the chief servant in the household and it showed who he was and what his position within the household was.
There does not have to be a deeper meaning in this respect.
It would be like a modern name tag on the garment in order to identify the owner.