Basically, it depends on the product. Meat has to be slaughtered and prepared by observant Jews. Grape Juicejuice or wine can only be touched by observant Jews before pasteurization. Certain cooked foods require an observant Jew to perform the cooking (or according to Ashkenazi custom, to turn on the fire before cooking),etc etc. In these cases, knowing the ingredients is not enough, one also needs to know that the proper rules were followed.
Some foods do not have these requirements, and one just needs to ensure that all the ingredients are kosher. However, many foods have ingredients that are not listed explicitly on the label (often in the context of catch-all terms like "flavorings"). Some ingredients (e.g. glycerin) can be made from both kosher and non-kosher sources. It requires expertise in kashrus to know which kinds of foods can be determined to be kosher from the label and which can't.
In my personal experience, rabbis involved in kashrus are willing to look at a label. Sometimes they say the product is kosher, and sometimes they tell you to verify some information with the company that makes it. And sometimes they tell you you need to get product with a hechsher.
Certain products are known to not be problematic and require no hechsher, such as unflavored water and seltzer, extra-virgin olive oil, etc.
And some products might not need a hechser but require examination at home, such as many vegetables that need to be checked for insect infestation.