After the destruction of the first temple, it had to be rebuilt from scratch. When Herod rebuilt the temple on the advice of Bava be Buta (Bava Basra 3b), it was considered as re-establishing and remodeling the existing temple established by Ezra. Thus, it was considered as part of the Second Temple, rather than a different building. During the entire construction, services continued and it was regarded as the Temple continuing to exist.
Ezra's Temple, Herod's Temple and Ezekiel's vision of the Third Temple
All our systems, processes, and institutions were born in the shadow
of the destruction of the Second Temple. So many rabbis and millions
of other Jews had been slain in their prime by Rome, that knowledge of
the Second Temple was lost and had to be recovered by deductive
reasoning and homiletic interpretation of scriptural verses, resulting
in disagreement about most Temple related matters. This has led to
great confusion about the Temple, even among some of the greatest
scholars.
To set the record straight: there were two Second Temples. The first
one was built 70 years after the destruction of Solomon's original
Temple. This second Temple stood for 332 years, then was removed and
rebuilt by Herod. Herod's Temple took eight years to build and stood
for 80 years, then was destroyed by Rome. So different from each other
were these two Temples that the Medrash (Numbers Rabba 14) assigns a
different sacrifice as their essence. The original Second Temple,
built by Zerubavel, governor of Judea, under the direction of Ezra and
King Darius of Persia, is represented by a ram as a burnt offering.
The rebuilt Second Temple, erected by Herod, is represented by a goat
as a sin offering.
For the most part, the Talmud treats the two Temples as one, and we
often think we are learning about Ezra's Temple, when in reality we
are almost always learning about Herod's Temple.
Few people realize that the miracle of Chanukah took place in Ezra's
Temple, not Herod's. The Maccabees vanquished the Greeks and
rededicated Ezra's Temple in 3597 (163 B.C.E), fully 145 years before
Herod's Temple was built.
The Mishna and the Rambam describe only Herod's Temple. Even the great
Rambam confuses the two, stating explicitly in the Mishneh Torah (Laws
of the Temple) that the height of Ezra's Temple was 100 cubits, but in
truth only Herod's Temple was 100 cubits high. Ezra's Temple was 60
cubits high as reported in Ezra 6:3. In Herod's speech to the Jews,
recorded by Josephus (Antiquities, book XV), Herod proclaims that the
original builders of the Second Temple were limited by the Persian
king to a Temple only 60 cubits high, while he, Herod, promised to
raise it to a the appropriate height of 100 cubits.
This Temple of Herod was no simple beautification project. Herod
removed Ezra's Temple, stone by stone, right down to the ground, and
then removed the foundations and built an entirely new Temple of his
own. Herod enlarged the Azora (Inner Courtyard) which was forbidden by
Halacha without a Sanhedrin of 71 judges, a Jewish King, and the Urim
and Tumim (the oracle of the High Priests Breastplate). Herod, who was
not Jewish, had murdered all the members of the Sanhedrin; the Urim
and Tumim had not existed since the destruction of the First Temple.
In effect, the Second Temple described in the Mishna and the Rambam
was an illegal structure, doomed to destruction from the very day it
was built.