Here are some sources that come from two Otzar Hachochma forum pages (here and here).
Yagid Davar points out 3 quick finds:
Divrei Hayamim 2 5:12 says that they wore Butz (linen):
וְהַלְוִיִּ֣ם הַמְשֹׁרְרִ֣ים לְכֻלָּ֡ם לְאָסָ֡ף לְהֵימָ֣ן לִ֠ידֻתוּן וְלִבְנֵיהֶ֨ם וְלַאֲחֵיהֶ֜ם מְלֻבָּשִׁ֣ים בּ֗וּץ בִּמְצִלְתַּ֙יִם֙ וּבִנְבָלִ֣ים וְכִנֹּר֔וֹת עֹמְדִ֖ים מִזְרָ֣ח לַמִּזְבֵּ֑חַ וְעִמָּהֶ֤ם כֹּֽהֲנִים֙ לְמֵאָ֣ה וְעֶשְׂרִ֔ים [מַחְצְרִ֖ים] (מחצררים) בַּחֲצֹצְרֽוֹת׃
also the Levites who were the singers, all of them, even Asaph, Heman, Jeduthun, and their sons and their brethren, arrayed in fine linen, with cymbals and psalteries and harps, stood at the east end of the altar, and with them a hundred and twenty priests sounding with trumpets — (JPS translation)
The Chasam Sofer suggests that they had special clothes made from Techeiles and Argaman (dyed wool).
Rambam in Hilchos Klei Hamikdash Vehaovdim Bo 10:13 says that Leviyim wore an Efod Bad (linen):
זה שאתה מוצא בדבר נביאים שהכהנים היו חוגרין אפוד בד לא היו כהנים גדולים שאין האפוד של כ"ג אפוד בד ואף הלוים היו חוגרין אותו שהרי שמואל הנביא לוי היה ונאמר בו נער חגור אפוד בד אלא אפוד זה היו חוגרים אותו בני הנביאים ומי שהוא ראוי שתשרה עליו רוח הקדש להודיע כי הגיע זה למעלת כהן גדול שמדבר על פי האפוד והחשן ברוח הקדש:
The statements found in the words of the prophets that the priests would wear an ephod of linen does not mean that they were High Priests. For the High Priest's ephod was not of linen [alone]. For the Levites would also wear such a garment, for the prophet Samuel was a Levite, and [I Samuel 2:18] describes him as "a youth, girded with a linen ephod." Instead, this ephod was worn by the students of the prophets and those who were fit to have the Holy Spirit rest upon them to make it known that such a person reached a rung equivalent to that of the High Priest who speaks with the Holy Spirit via the medium of the ephod and the breastplate. (Chabad Translation)
Baberacha Hameshuleshes referenced this article (link from Yagid Davar), which discusses this at length.
Ma Shenachon Nachon brings the Shem Mishmuel (Parshas Vayakhel) who says that the Leviyim had no special clothes.