Your observation - of the missing wife in the verse - is the basis for the halacha that one may not get married during Chag.
See the Torah Temima and the Rabeinu Bachyeh on the verse that mention this.
Regarding the actual question, see the Ibn Ezra in verse 14:6 who mentions that you and your sons have an obligation to go to the Mikdash on the 3 festivals, whereas your wife and daughters do not.
אתה וביתך. אתה ובניך חיוב וביתך רשות כי מצות שלש פעמים על הזכרים והאשה ההולכת למקום הנבחר כנשי אלקנה יש להן שכר:
You and your household you and your sons are obliged, whereas the rest of your household are permitted. The commandment to observe the three festivals is binding only upon males [Exodus 23: 17]; but those women will be rewarded who, like the wives of Elqanah, [I Samuel 1: 3 ff .], go on pilgrimage to the appointed place.
So possibly the verse you mentioned excluded the wife, not because you don't have an obligation to rejoice with her, but because she may not be around, since she may have opted to remain at home - yet this wouldn't give you an excuse to remain with her.