Something you have to understand is that, although קמץ קטן and קמץ גדול are written the same and have the same name, this is due to historical reasons, not because they’re the same vowel. קמץ קטן is the “short vowel” of חוֹלָם, and קמץ גדול is the “long vowel” of פתַּח.
In the Hebrew of the Chumash, a short vowel can only appear in a closed syllable (except for חטף vowels, but that’s another subject).
A closed syllable is one that ends in a consonant, instead of a vowel.
For example, the word כֻּלָּם starts with a closed syllable and ends in another closed syllable: /kul-LAM/.
The word גֻּדְלוֹ also starts with a closed syllable, but ends in an open one: /gud-LO/.
In other words: A closed syllable is one in which the vowel is followed by either a דגשׁ חזק, or a silent שווא.
קמץ קטן is a short vowel. As mentioned above, a short vowel can only appear in a closed syllable. Therefore, a קמץ קטן can only appear if it is followed by a דגשׁ חזק or a שווא נח. However, this doesn’t mean that every such קמץ is קטן; only that it MIGHT be.
The געיא/מתג is a symbol which fundamentally means that a vowel is longer than usual. Sometimes, this is due to stress; sometimes, it’s for musical purposes. Other times, the געיא is used to show that a vowel is long instead of short.
For example, take the word שָׁמְרָה. It starts with a closed syllable containing a קמץ of some sort. Normally, without any reason to believe otherwise, we would read it as /shom-RA/; and indeed, in most cases this is the word’s pronunciation.
However, in some parts of תנ״ך, a געיא is attached to the first קמץ. This means that the קמץ here is a long vowel, not a short one. And, as mentioned in the first paragraph, קמץ קטן is a short vowel, while קמץ גדול is a long one. Therefore, this קמץ is a קמץ גדול, and the word is pronounced /sha-m’RA./
Conclusion: קמץ is קטן in a closed syllable if it is not attached to a געיא.
Regarding the other בעל קריאה who told you that קמץ קטן appears in words with roots that have a חוֹלם—he’s right, but what he told you isn’t really a rule.
Of course, every rule has a few exceptions, but these are only due to historical linguistic evolution, and don’t arise from any pattern. In almost every single case, the above rule applies.
Note: A Kamatz at the end of a word is never קטן, UNLESS it is connected to the next word by 1) being penultimately stressed and having a conjunctive trope, or 2) being connected to the next word by a מקף (־).