The Mishnah Berurah there explicitly addresses your question. In s.k. 28 he writes:
ואפילו לאותן המניחין תפילין בחוה"מ בלי ברכה או המניחין תפלין דר"ת אחר שחלץ תפלין דרש"י או שחלץ תפלין ע"מ להחזירן דהרמ"א בסי"ב פסק דא"צ לחזור ולברך כשמניחן אח"כ אפ"ה עבירה היא להסיח ביניהן דלכתחילה בעינן שיהיו סמוכין ותכופין זה לזה דכתיב והיה לך לאות על ידך ולזכרון בין עיניך שיהא הוייה אחת לשתיהן
Even those who wear tefillin on Chol HaMoed without a blessing, or those who wear Rabbeinu Tam tefillin after they've already worn Rashi's tefillin, or someone who takes off tefillin with intention to put them back on (which the Rema in Seif 12 rules that one doesn't make another blessing) and puts them on later, even so it is a transgression to speak between them. This is because ideally we need the two tefillin to be connected and adjacent to each other, as it is written: "It shall be for you as a sign on your arm and a remembrance between your eyes", that they should have the same "being" for the two of them.
In Shaar HaTziyun he cites the Magen Avraham (s.k. 14):
ואפי' לאותן המניחין בח"ה בלא ברכה אפי' הכי עבירה היא להסיח (ב"י מהרא"י)
Even those who wear tefillin on Chol HaMoed without a blessing, even so it is a transgression to speak.
The Magen Avraham cites the Beis Yosef (ad. loc.) and Terumas HaDeshen (#107) (who the Beis Yosef brings). The Pri Megadim (Mishbetzos Zahav ad. loc. s.k. 4; Eishel Avraham ad. loc. s.k. 17) explains that this concept of having the two tefillin be connected comes from Ba'al HaMeor at the end of Rosh Hashanah. According to him, not only is speaking forbidden, but even answering Amen Yehei Shmeih Rabbah.
However, the Pri Megadim infers from the Shulchan Aruch in Seif 10 that the latter disagrees with the Ba'al HaMeor. Really the issue is unnecessarily causing a new blessing to be said. Therefore, someone who doesn't make a blessing on Chol HaMoed could in fact answer Yehei Shmeih Rabbah. The Magen Avraham (s.k. 17) therefore says that it would be permissible to respond in such a case:
ומ"מ משמע דהמניח בחה"מ בלא ברכה רשאי לענות דליכא איסורא בזה אלא משום שגורם ברכה:
It would seem though the the Magen Avraham holds לכתחילה, ideally, to consider the opinion of the Ba'al HaMeor. That's why he forbade speaking on Chol HaMoed between the tefillin. Indeed, the Mishnah Berurah above explained the reasoning for not interrupting the same as the Ba'al HaMeor.
Therefore, even though the Mishnah Berurah (s.k. 36) rules like this latter Magen Avraham that it's permissible to respond to Amen Yehei Shmeih Rabbah on Chol HaMoed between the tefillin, he says to move the tefillin shel yad from their spot and then put them back before putting on the shel rosh. This way it'll be as if the act of putting on the two tefillin was connected:
ובחוה"מ או המניח תפילין דר"ת וכיוצא שמניח בלי ברכה יש להקל ולהפסיק לעניית איש"ר וקדושה וברכו ואמן אך אח"כ יזיז הש"י ממקומו קודם שיניח הש"ר כדי שיהיה הוייה אחת לשניהן:
For some reason though the Magen Avraham and Mishnah Berurah don't want to rely on this solution for trivial interruptions. They only permit these important responses.