Going out on a limb here. The sister sugya to yours is in Sanhedrin 20a. There the Yad Ramah says that 'this practice is similar to the people who set up a table for Gad (which is assur) except that there, they have intention of worship while here it is only for a good siman'. What i want add is an explanation of 'siman' based on the gemara in Horios 12a concerning simana milsa and the simanim on the night of Rosh Hashana where a given idea is started and this is a siman, a cause, to be continued. If the bracha/mazal which you are looking for is the blessing of neatness, then this is an option for a source. But it seems this pshat is not working with the Gad mazal of good fortune. For more pshatim see Margolios HaYam there, but nobody he brings says quite like what your magid shiur said.