Can one go to a sefroim store with the intention of not buying a Sefer and look into a sefer and read some of its contents? Is it considered stealing?
See this article which cites mainly Choshen Mishpat for support.
In short - browsing is permitted as the store owner expects customers to browse. If, at the end, you don't buy because you didn't find what you wanted or didn't like the price, etc. that's fine. A nicer thing to do is to indicate this to the owner, but, it's not required.
However, if you have no intention to buy anything at all, this is a violation of oppressing your neighbor ("ona'ah), since the owner expects you to buy something, and you may be distracting him form attending to other serious customers.
An exception is if you walk into a very large or crowded Seforim store where the owner is unaware of individual customers entering or leaving. In that case, as long as you don't disturb the owner or his workers, you are fine, as the owner has no expectations.
Another possibility would be if the owner asks, "Can I help you" and you answer "I'm just browsing, I may not actually buy anything" and the owner is fine with this. In that case, the owner has given you permission not to buy and he clearly indicates that you have no obligation to do so.
You can also assist your friend who is buying, even if you're not. However, this only allows you to read books that your friend intends to buy.