Most of it wasn't kept in the Beis Hamikdash and the part that was in the Bais Hamikdash was kept in hidden places.
The Rambam in Hilchas Para Aduma Perek Gimmel says
אֵין מַכְנִיסִין כְּלוּם מֵאֶפְרָהּ לְהַנִּיחוֹ בָּעֲזָרָה שֶׁנֶּאֱמַר (במדבר יט ט) "וְהִנִּיחַ מִחוּץ לַמַּחֲנֶה". וּשְׁלֹשָׁה חֲלָקִים הָיוּ חוֹלְקִין אֶת כָּל אֶפְרָהּ אֶחָד נִתַּן בַּחֵיל וְאֶחָד בְּהַר הַמִּשְׁחָה וְאֶחָד מִתְחַלֵּק לְכָל הַמִּשְׁמָרוֹת. זֶה שֶׁמִּתְחַלֵּק לְכָל הַמִּשְׁמָרוֹת הָיוּ הַכֹּהֲנִים מְקַדְּשִׁין מִמֶּנּוּ. וְזֶה שֶׁנִּתַּן בְּהַר הַמִּשְׁחָה הָיוּ יִשְׂרָאֵל מַזִּין מִמֶּנּוּ. וְזֶה שֶׁנִּתַּן בַּחֵיל הָיָה מוּכָן וּמֻצְנָע שֶׁנֶּאֱמַר (במדבר יט ט) "וְהָיְתָה לַעֲדַת בְּנֵי יִשְׂרָאֵל לְמִשְׁמֶרֶת" מְלַמֵּד שֶׁמַּצְנִיעִין מִמֶּנּוּ. וְכֵן הָיוּ מַצְנִיעִין מֵאֵפֶר כָּל פָּרָה וּפָרָה שֶׁשּׂוֹרְפִין בַּחֵיל. וְתֵשַׁע פָּרוֹת אֲדֻמּוֹת נַעֲשׂוּ מִשֶּׁנִּצְטַוּוּ בְּמִצְוָה זוֹ עַד שֶׁחָרַב הַבַּיִת בַּשְּׁנִיָּה. רִאשׁוֹנָה עָשָׂה משֶׁה רַבֵּנוּ. שְׁנִיָּה עָשָׂה עֶזְרָא. וְשֶׁבַע מֵעֶזְרָא עַד חֻרְבַּן הַבַּיִת.
וְהָעֲשִׂירִית יַעֲשֶׂה הַמֶּלֶךְ הַמָּשִׁיחַ מְהֵרָה
None of its ashes are brought into the Temple Courtyard for storage, as ibid.:9 states: "And he shall place it outside the camp." The ashes were divided into three portions: one was placed in the chayl one, on the Mount of Olives, and one, was divided among all the priestly guardposts.
The one that was divided among all the priestly watches was used by the priests to sanctify themselves. The one that was placed on the Mount of Olives was used by the entire Jewish people for sprinkling. And the one that was placed in the chayl was prepared and hidden away, as implied by ibid. which states: "It will be a security for the congregation of Israel." This teaches that it was put away for safekeeping.
Indeed, in the chayl, they would put away for safekeeping a portion of the ashes from every red heifer that was burnt