Although I am not Jewish, I have a keen interest in the Jewish Scriptures, and am currently looking at the building of the Temple by Solomon. In the book of Chronicles, it records how King David established the 24 divisions of musicians from the families of Asaph, Jeduthun and Heman (ch 25) as well as assigning other functions to the Levites (in ch 26). However, apart from one possible reference in 2 Chron 23:6 (when Joash is made king and a distinction is made between 'the priests and the ministering Levites' who enter the temple - although this could be a reference to the temple courts - v13), there seems to be no references to the Levites serving within the temple building itself. Should it be assumed that they only ever served in the temple courts?
No one was allowed to enter the Sanctuary itself unless they were a Kohen who had washed his hands and feet, and only for the purposes of service (usually bringing incense or lighting the menorah).
Male Israelites and Levites were permitted to enter the easternmost portion of the Temple Court (the Ezrat Yisrael) if they were completely Tahor (pure). They could enter further into the court (but not the Sanctuary) to bring a sacrifice (if an Israelite or a Levite) or to sing and play instruments during the service (if a Levite). This may be what the passage is referring to.
Also, there was a large courtyard (the same size approximately as the Temple Court, called the Women's Courtyard) directly to the east. Anyone could enter there if they were pure, and the Levites would sing there when the sacrificial service was not occurring. So this is another possible explanation of the passage.
The regulations of who may enter various areas of the Temple are in Mishna Kelim Chapter 1.