There was most likely no such rope.
The source of this idea is two mentions in the Zohar:
The Zohar on Parashat Acharei Mot (67a) describes the Kohen Gadol’s preparation before entering the Kodesh Hakodashim and states that “a gold chain was tied to his leg,”
In the Zohar on Parashat Emor (102a) the description of the Kohen Gadol’s entry into the Kodesh Hakodashim includes this statement: “Rav Yitzchak said: ‘One rope was tied to the Kohen’s leg when he went in, so that should he die there they could pull him out.’”
But the many other sources make no mention of a rope:
Rambam does not mention it in Yad Hachazakah nor do any other Rishonim discuss it. It is not mentioned in the Ashkenazic Avodah piyyut “Amitz Koach” (circa tenth century) nor in the older Sephardic Avodah piyyut “Atah Konanta.” Moreover, it is worth noting that the rope is not mentioned in the Apocrypha, the Dead Sea Scrolls or the pseudepigrapha. Finally, one of the best eyewitnesses of Beit Hamikdash activities, Josephus, fails to note the existence of such a rope.
And some sources relate events that imply there was no rope:
The Gemara (Yoma 53b) relates that there was once a Kohen Gadol who recited a long prayer while in the Heichal (main sanctuary building). His fellow Kohanim assumed that he had died, and decided to enter the Heichal to look for him (obviously, the implication being there was no rope)
Rabbi Chiya relates (Yoma 19b) that once when a Tzeduki Kohen Gadol was still in the Heichal, those outside in the Azarah (Temple courtyard) heard a sound, and assumed that an angel had hit him in the face. They entered the Heichal and found him dead. Again, the implication is that there was no chain attached to this Kohen Gadol.
There also are halachic problems with the rope. It could constitute an extra garment invalidating the service. If placed under the Kohen Gadol's clothes, it would be a chatzitza; if over, it would impede his movements.
If the rope is a gold chain rather than an ordinary rope, there are further halachic problems, such as spreading tumah and wearing gold on Yom Kippur.
Then why does the Zohar mention a rope or chain? Rabbi Uri Sherki explains it as allegorical:
the Kohen Gadol, upon entering the Kodesh Hakodashim on Yom Kippur, can become so overwhelmed by the spiritual that he is liable to “forget” to leave. The Zohar uses the imagery of a “rope” as a reminder to the Kohen Gadol that the Jewish people need him and he must “pull” himself back to this world after the extraordinary otherworldly experience of Yom Kippur.