Your citation of the Derashos HaRan is incomplete. Here is the text of the entire paragraph:
ואמנם יתאמת שמי שישען עליו באמת להיות (נשען) [נכנס] בעובי הקורה
בעבודתו יתברך שיעשה מעשה יותר רחוק מן הטבע [יורה על היותו נשען על השם
יתברך באמת] והיותר רחוק מן הטבע הוא מעשה אברהם אבינו עליו השלום בעקידת
יצחק כי לא היה נמשך לו עונש כלל אם לא יעקדנו כי לא צוהו השם יתברך בזה
והנה הבטיחו כי ביצחק יקרא לך זרע וזהו לשון הכתוב קח נא את בנך וגו'
וכבר נודע שזה המאמר אינו לשון צווי אלא לשון בקשה שהראה לו שייטב בעיניו
אם ימחול אברהם על הבטחתו ויעקוד בנו ולו היה אברהם משיבו הן לי לא נתתה
זרע כי אם זה והבטחתני בו ואיך אעשה זה לא נחשב לו עון ולא ישיגנו עונש
מזה ואף על פי כן לגודל אהבתו את השם יתברך מצא את לבבו לעקדו כדי שיעשה
חפץ השם יתברך לבד ואף על פי שאילו לא עשאו לא היה משיגו עונש כלל וזהו
ענין העקידה וסודו
The Ran is defining the category of "ישען עליו באמת" "relying on Him (God) in truth". He says that the epitome of this category is doing something that is "יותר רחוק מן הטבע" "most distant from nature", i.e. the most unnatural thing for a human to do. The greatest example of this, he says, is Avraham performing Akeidas Yitzchak. This is so because slaughtering one's child is incredibly painful and difficult. Had the situation been one in which Avraham would have been punished for not complying with the request, it would be less incredible that Avraham did it. Since, however, Avraham had the option to "opt out" at any time with no consequences, the fact that he went through with it is so much more incredible and it demonstrates an unparalleled love for God.
According to this understanding, the test is made harder by the fact that it would have been so easy for Avraham to not go through with it.
How does this relate to the general concept of "gadol hametzuveh v'oseh"? If your question is simply how Akeidas Yitzvhak can be greater than the other tests if it was not commanded and they were commanded**, (and this is the implication of your final paragraph) I would say that this is likely a misunderstanding of "gadol hametzuveh v'oseh". What this concept means is that If Person A is commanded to do XYZ and Person B is not commanded to do XYZ, A's performance of XYZ is greater than B's performance of XYZ. That does not mean, however, that if A is commanded to do XYZ and B is not commanded to do ABC that A's performance of XYZ is greater than B's performance of ABC, because it is possible that ABC is intrinsically greater than XYZ. So in this case the non-commanded Akeidas Yitzchak could still be greater than the other commanded tests because the act itself is intrinsically greater than the other acts. (In other words "gadol hametzuveh v'oseh" is only applicable in a comparison of two identical acts.)
However, if your question is (and this is the implication of the title) why the non-commandedness of Akeidas Yitzchak would increase its greatness relative to itself (i.e. that Akeidas Yitzchak being non-commanded is greater than Akeidas Yitzchak if it would have been commanded) the above does not suffice.
To address this I would say that it depends how you understand the reason for "gadol hametzuveh v'oseh". In this answer I quoted the opinion of Tosafos that it is because doing an obligated mitzvah involves more anguish than doing a voluntary mitzvah because when doing a voluntary mitzvah you know that you can always give up, and the first opinion cited by the Ritva that obligatory mitzvos are harder due to Satan's machinations. Indeed these explanations would seem to be contradictory to the Ran who appears to be saying the opposite – that being voluntary makes it harder.
However, the second answer cited by the Ritva (in the name of the Ramban) is that it is greater when one is fulfilling the King's command than when one is not fulfilling the King's command. I think this could fit with the Ran's explanation here. A general example of "einah metzuveh v'oseh" is someone that was never asked by God to so something at all (e.g. women performing time-bound positive commandments). That would fall under the category of not fulfilling the King's command because it is purely voluntary. By Akeidas Yitzchak it was not purely voluntary; God requested that Avraham do it but just didn't assign a punishment if he wouldn't do it. In this case he was directly fulfilling the request of the King. In other words, it is possible that despite everything the Ran said he would still consider this a case of "metzuveh v'oseh" and not "einah metzuveh v'oseh", and thus no contradiction arises.
A further point is that perhaps the Ran understands the rule of "gadol hametzuveh v'oseh" as referring to general mitzvos. This is because in general mitzvos are not designed to be specifically cruel to the performer. Therefore it is not inherently super-difficult to perform the mitzvah; adding the obligatory nature would make it harder in accordance with Tosafos and Ritva 1. But Akeidas Yitzchak is by design fundamentally difficult, perhaps even cruel. Avraham is being asked to slaughter his own child. In that case it is precisely the voluntary nature that makes it harder, because he had the ability to easily avoid causing excruciating pain to himself by simply opting out. In other words, it is possible that in general being obligatory makes something harder but in unique situations being voluntary makes something harder.
Summary of three main points
- "Gadol hametzuveh v'oseh" is only applicable in a comparison of two
identical acts.
- It is possible that despite everything the Ran said he would still
consider this a case of "metzuveh v'oseh" and not "einah metzuveh
v'oseh", and thus no contradiction arises.
- It is possible that in general being obligatory makes something
harder but in unique situations being voluntary makes something
harder.
**This is not even taking into account that some of the other tests were commanded. In fact R. Yoel Stanitzky argues (Peri Mussar, Ma'amar Asarah Nisyonos) that Akeidas Yitzchak was a bigger test than getting thrown into the furnace, precisely because the former was commanded while the latter was not.
