Rashi quotes the machlokes between Rav and Shmuel whether it was the physical plant or the number of years that the second temple existed. While some people say that Chagai means to hint about the temple to be built by the mashiach, most say that this refers to the second temple which was about to be built (since Chagai, Zechariah, and Malachi were part of the Anshe Knesses Hagedolah) and that before it would be destroyed, this is what it would become.
Note that Rashi comments on the phrase "will be greater". That is Rashi says that Rav holds that the actual temple itself (after Herod rebuilt it) was "more magnificent" than the first temple. Shmuel holds that the "greater glory" was that it lasted longer. Both speak of it even though it did not have the same spiritual glory as the first temple.
The fame of the temple spread throughout the Roman Empire which could also mean that the glory "was greater" in that it was a major attraction over much of the known world.
ט גָּדוֹל יִהְיֶה כְּבוֹד הַבַּיִת הַזֶּה הָאַחֲרוֹן מִן הָרִאשׁוֹן
אָמַר יְהֹוָה צְבָאוֹת וּבַמָּקוֹם הַזֶּה אֶתֵּן שָׁלוֹם נְאֻם יְהֹוָה
9 The glory of this last House shall be greater than the first one,
said the Lord of Hosts. And in this place I will grant peace, says the
Lord of Hosts.
Rashi: shall be greater: There was a controversy between Rav and
Samuel. One said: In the building; and one said: In the years, that
the years of the First Temple were four hundred and ten, and those of
the Second Temple were four hundred and twenty.
The Jewish Virtual Library says that not only was it a magnificent building, but that multitudes came to see it. I have seen references that not only Jews, but nonJews from all over the Roman Empire came to see the temple.
The Second Temple
The Second Temple was not only awe inspiring because of its religious
significance, but also for its physical dimensions, its grandeur and
its beauty. Thus, as the Roman generals sat surveying Jerusalem and
considering the Temple’s future they hesitated before ordering its
destruction. Jews, from that day to this, have yearned and prayed for
its rebuilding, and tourists and religious people alike have come to
behold the site on which it once stood.
Normally a city of 100 to 200 thousand people, three times a year on
the pilgrim festivals of Passover, Pentecost and Tabernacles,
Jerusalem’s population swelled to 1 million souls (the exact number
depending on the source of population estimates). On these occasions
this small ancient city had to cope not only with the throng of people
but also their sacrificial animals and offerings, necessitating
temporary increases in food supplies, accommodation, ritual bathing
facilities, and all aspects of commerce. It was Herod, installed by
the Romans as governor of Jerusalem, who faced these logistical
problems, and who consequently set about renovating the city and the
Temple to accommodate this massive periodic influx.
Before work began on the Temple, Herod spent eight years stockpiling
materials for its construction. Then, a workforce of over 10,000 men
began its construction including a contingent of 1,500 specially
trained priests who were the only ones permitted to work on the
innermost and holiest parts of the Temple. Building continued for a
further twenty years, though the Temple was in a sufficiently ready
state within three and a half years of its commencement to be