While I'm loathe to get involved with transliterations here, "geh" looks much closer to accurate to me than "gay".
The general rule in Tanakh is a letter is only pronounced if it has a vowel mark. The exception to that rule is if it is a sheva nach on the last letter of the word, where it is usually omitted as obvious.
In your case we have a use of a י as a mater lectionis to represent the segol vowel, like in other words עיניך אליך לפניך עליה תראינה. The י is not pronounced in any way. If it were pronounced, like in the word לילה, there would be a sheva mark under the י.
This is the only place in Tanakh where the segol vowel is written plene not as part of a suffix, so it does throw people off and many don't notice the difference with the more common conjugation גַּיְא or גֵּיא. (There are some variant manuscripts that do include a sheva under the י here, but they are a minority. The Aleppo Codex and related manuscripts do not include a sheva there.)
See this article by Adam Bin-Nun about an extensive debate in Yemen 100 years ago how to properly pronounce this word.