In ספר אם למקרא ולמסורת, he gives the following general rules:
- In general, 'ah'.
- If the next word starts with an א or ר, it'll be a קמץ I.e. 'o' or 'oh' depending on your מנהג about קמץ קטן. (In the 'סימנים' version of the תיקון קוראים, they clearly show the difference between a קמץ גדול and קטן.)
- If it is an ע, in general it will be a קמץ but if the the ע of the next word has a קמץ גדול and the stress of the word is not on that ע, it will be a סגול I.e. 'eh'
- If it is a ה or ח, in general it will be פתח I.e. 'ah' but if the next word starts with a קמץ גדול it will be 'eh' and if it is a ח with a חטף קמץ i.e. the קמץ with two dots next to it underneath the ח, it will be an 'eh' too.
- If it is a ה meaning 'the' it will be a קמץ.
- Before a י with a שווא (the two dots underneath), it will be 'ah'
- If it is not connected to any other words or is connected to the previous word as opposed to the next, it should be קמץ
I am not sure if these are iron clad rules without any exceptions but they are quite comprehensive.
The word במה has different rules. You'll never find it as 'b'ma' meaning שווא then פתח. So that is a separate question to the word מה, somewhat surprisingly.
It seems from a quick look through תנ''ך that it is 'oh' i.e. a קמץ when the word following במה starts with an א or ה or when במה is a stand alone word, but bameh in all other places.
The part about the א is actually found in the מלאכת שלמה on the משנה at the beginning of the second פרק or שבת which is במה מדליקין. (Oddly, all the versions of משניות I saw had bameh for both of the במה in במה מדליקין whilst all the סידורים had bamo.
The example before the ה is in שמואל א פרק יד פסוק לח.