Chametz is prohibited to eat or own on Passover, and this includes any flour made of the five grains, Wheat, Rye, Spelt, Barley, and Oats, that has come into contact with water for enough time to halachically ferment, which is a period of at least 18 minutes.
Se'or, which is the heavily-leavened sourdough that was commonly used as a leavening agent, is also prohibitted to eat or own on Passover. The biological yeast that we use today for leavening bread is itself not prohibited, but may not be added to any flour mixtures, as it speed the fermentation process and is likely to become chametz. See here.
In general, fermented products such as alcoholic beverages are permitted on Passover so long as they are not made from the grains we are worried about. Alcohol is not prohibited on Passover. In fact, we are required to drink four cups of wine on the first night at the seder.
Practically speaking, one should only buy and consume products on Passover if they are marked with a legitimate Passover kashrut symbol.
In regard to why we eat unleavened bread, or matzah, on Passover, and what it represents, the mishna, cited famously by the haggada, explains that it symbolizes the haste in which the Jews left Egypt, so much so that they would not even wait for their bread to rise. Other symbolic reasons are given, such as it represents the deflation of the human ego, or that it represents complete freedom in its ultimate simplicity.