In my experience having "passuled" more Torahs than I should admit over the last 13 years layning in Shuls in the US and Israel, mistakes are mistakes, whether they are noticed or not.
However, in regards to the simple Halacha, if the mistake does not seriously affect the letter's form, then the Torah is still Kosher, but should be corrected as soon as possible. If it's a serious mistake, you should say something (such as if two letters are clearly linked to each other or a letter has faded to the point that it looks like something else.)
Personally, I tend to not look for mistakes, but if I see one, I say something. I do that mainly for Tircha D'Tzibura purposes - it's really annoying to the congregation if the Ba'al Koreh keeps "passuling" the Torahs and stretching out davening more than necessary. Also, I find it helpful to know what the letters are supposed to look like.