The Midrash rebukes Avraham for swearing to Avimelech and making a covenant (Genesis 21:23-32):
אָמַר לוֹ הַקָּדוֹשׁ בָּרוּךְ הוּא אַתָּה נָתַתָּ שֶׁבַע כְּבָשׂוֹת בְּלִי רְצוֹנִי, חַיֶּיךָ שֶׁאֲנִי מַשְׁהֶה בְּשִׂמְחַת בָּנֶיךָ שִׁבְעָה דוֹרוֹת. אַתָּה נָתַתָּ לוֹ שֶׁבַע כְּבָשׂוֹת בְּלִי רְצוֹנִי, חַיֶּיךָ כְּנֶגֶד כֵּן הוֹרְגִים מִבָּנֶיךָ שִׁבְעָה צַדִּיקִים
Hashem said to [Avraham]: "You gave seven sheep without my consent, by your life I'll delay the joy of your children seven years. You gave seven sheep without my consent, by your life, correspondingly, seven of your righteous descendants will be killed...etc.
We see Yitzchak did similarly (Genesis 26:28-33). Are there sources which rebuke him similarly, or those which explain why it was fine for Yitzchak but not Avraham?
The Midrash does explicitly mention the seven sheep, which weren't part of Yitchak's oath, but the Mesillas Yesharim (Chapter 4) says the issue was specifically the covenant.