Well you're apparently not alone in thinking that it's not obvious that the Israelites said something of this sort in 5:21. For instance, Ibn Ezra in his commentary to 14:12 says that they must have said this at some point but we don't find it recorded anywhere. He opines that it is included in 6:9 where the verse says that "they didn't listen to Moses". Ibn Kaspi (commentary to 14:12) elaborates on this slightly, noting that "they didn't listen to Moses" is The Torah recording the incident in general terms without bothering to give all the details. (The details being what we find out in 14:12, namely, that the people said that they would rather be slaves in Egypt than die in the wilderness.)
Interestingly, the Samaritan Pentateuch's version of 6:9 has an additional clause in which it is explicitly stated that the Israelites told Moses that they would rather stay as slaves in Egypt than die in the wilderness:
וידבר משה כן אל בני ישראל ולא שמעו אל משה מקצר רוח ומעבדה קשה ויאמרו אל משה חדל נא ממנו ונעבדה את מצרים כי אוב לנו עבד את מצרים ממותנו במדבר
And Moses spake so unto the children of Israel: but they hearkened not unto Moses for anguish of spirit, and for cruel bondage. And they are saying to Moses: leave us please and we shall serve to Egyptians because it is good for us to be servants of Egyptians than to die in the wilderness. (STEP translation, my emphasis)
Whether this is due to a differing original text or a later interpolation, it apparently agrees with Ibn Ezra and Ibn Kaspi about when the Israelites actually said this.
R. Ephraim Luntschitz (commentary to 14:12) acknowledges that Rashi's explanation requires elaboration. He explains that when the Israelites in 5:21 said "to give a sword in their hands to kill us" they meant that even if they would escape Egypt the Egyptians would chase after them and kill them in the wilderness. Thus, in 14:12 when the Egyptians have chased after them and are about to kill them, they tell Moses that this is exactly what they had said would happen back in 5:21.
Alternatively, when we look at Rashi's source in the Mechilta it appears that it is not actually saying that the Israelites said this in 5:21. Rather, in 5:21 the Israelites merely expressed their distress over the slavery. Later when many of their brethren died during the Darkness, that was even more distressing to them. Now when the Egyptians are about to kill them in the wilderness it is even more distressing than when their brethren died during the Darkness (because at least their brethren were properly buried and eulogized). Thus, it seems that 14:12 should be read as only the first half of the quote being an actual quote, while the second half is actually the explanation of the quote. That is to say, rather than reading it as we said "leave us be and let us serve Egypt, for it is better to serve Egypt than to die in the wilderness" we should read it as we said "leave us be and let us serve Egypt" because it would have been better for us to serve Egypt than to die out here in the wilderness.
This point is elaborated on by R. Elijah Mizrachi (supercommentary to Rashi 14:12), who notes that this has to be the case because at the time of 5:21 they were still in Egypt so it wouldn't have made sense for them to say then that they would rather be slaves in Egypt than die in the wilderness.
R. Samuel David Luzzato notes (commentary to 14:12) the Samaritan version which has the Israelites actually saying this in 6:21, but he argues that there would be no reason for them to have thought of dying in the wilderness when they were suffering the backbreaking labor in Egypt. And if they did somehow think of it then the verse would be inaccurate since it says that the reason why they didn't listen to Moses was due to the hard labor and their low spirits, when the real reason was that they made a calculated decision that slavery was better than death in the wilderness. He therefore also cites the Mechilta's interpretation, apparently understanding it to mean that the Israelites added in the second half of the statement now in 14:12 (that it would have been better to remain as slaves in Egypt).
