As a gentile, is okay to give a gift for Hanukkah to my friend's kids? I'm coming into this as a total newb so any suggestions are welcome.
Yes, and I'm sure your friend and their kids will much appreciate it!
Just use some common sense about it. Don't get them Christmas stuff, or anything from other religions. It's probably worth checking with the parents before giving something to make sure it's something that a) they're ok with b) the kids are interested in.
You may also want to look at Examples of considerations when bringing gifts to religious Jews.