Israel Yeivin, Introduction to the Tiberian Masorah (ed. E.J. Revell; Scholars Press, 1985) discusses these "large" and "small" letters (among other peculiarities pertaining to letters) on pp. 47-48 (§§ 84-85).
He refers to the Masorah's listing of "a few dozen" examples of large letters, although in the list provided in a previous answer there are 29 (so, two-and-a-bit dozen?). Only a few of these, he claims, allow for some kind of cogent explanation:
- those that begin a book (Bereshit, Shir HaShirim, Divrei HaYamim, Mishlei) or section (ס at Qohelet 12:13);
- those that fix attention on something of significance, e.g. Vayiqra 11:42 (half-way point in Torah counting letters) and 13:33 (counting words);
- providing a warning that "reading must be precise", as in Devarim 6:4.
But most, he believes, don't admit of any rationale (e.g. Bereshit 30:42; Devarim 29:27).
There are fewer examples of small letters, citing from the Masorah:
- three small final nuns: Yeshaya 44:14; Yirmiyahu 29:13; and Mishlei 16:28;
- others listed as: Bereishit 2:4; 27:46; Vayiqra 1:1; Devarim 32:18.
Finally, Yeivin also notes that such marking is not consistent across the older mss, e.g. Leningradensis having only three of each (large and small).