Chacham Ovadia writes in Yabia Omer 1:6(end of section 7) different opinions on listening to a woman's voice without knowing what she looks like. He brings the gemara in Berachos 57b that says: three things relaxs the mind of a person and one of them is kol (music) . Rashi says that it refers to a woman's voice. How can this be isn't a woman's voice erva? So he answers that this fits in well with the opinions that hold the voice of a penuya is mutar,but what should we say for the other opinons? So he answers that it is talking about when the person doesn't know or recognize the woman.
My question is isn't a better answer to say that Rashi is talking about the person's wife which is mutar and still hold that a penuya and even when you don't know or recognize the woman it will still be assur?