When you borrow something from the (secular) library does it follow the same halacha as a loan from one Jew to another?
As a general rule, in monetary matters, there is no difference between dealing with Jews or non-Jews; and in both cases the (written or oral) agreement between the parties is binding.
Notable exceptions are:
So in your case: all the rules of the library would apply and you would have to abide by them. However, if they make a mistake, then the rules in #2 above apply.
E.g. If you are late in returning a book, and they charge you a late-fee, and they err in the calculation, you do not have to correct their arithmetic, you can simply say "are you sure that's the amount?" and then pay the reduced fee.