I grew up hearing that if you put two hats or two yarmulkas at the same time that you could forget your learning- is there any source for this?
The Magen Avaham (OH 2:3) references Shaar HaKavanot of R. Hayyim Vital which states that one should not put on two items of clothes at the same time, as it can make one forget:
ויזהר מללבוש ב' מלבושים יחד בפעם א' כי קשה לשכחה (הכוונות)
One should be careful not to don two garments together at one time, for it can cause forgetfulness (Shaar HaKavanot).
This could perhaps be read to mean that one should not wear two garments at the same time. It seems likely that this is the source of the idea that you heard.
However, it doesn't appear (to me) that this is the Magen Avraham's intent, since then one would never be able to wear more than one garment at a time! Rather, it appears that he refers to donning two garment simultaneously; not wearing the two simultaneously.
Of course as with most things, proving the negative is very difficult, there could always be some source somewhere about wearing two hats, rather than donning two garments, but this seems (to me) to be the likeliest explanation.