There is no Jewish custom of celebrating on erev Rosh Hashana that I'm aware of. It's a day when many supplications are added to the prayers, when many people fast, and when we are all supposed to repent our misdeeds and make plans to do better in the coming year. Rosh Hashana itself, on the other hand, is a time when families get together for festive holiday meals. That is when people at apples and honey, say special blessings, etc. See this page for a good synopsis of Rosh Hashana observances (and this one for more on erev Rosh Hashana, though the latter page mentions several Chabad-specific customs).
Incidentally, although the question characterizes eating apples and honey as a "non-religious" custom, it is mentioned in halacha, which I think qualifies it as "religious".
Note that the evening preceding the day of Rosh Hashana, from sunset, is considered part of Rosh Hashana (and happens to be when we eat the apples and honey). The same is true in almost all cases that one is considering a day in Judaism: it starts at sunset.
I'm adding the following is in case I've misunderstood your question, and by "Erev Rosh Hashanah" you actually meant the night of (i.e., at the start of) Rosh Hashana: A family meal is good for the night of Rosh Hashana or its day (for lunch, say). Indeed, since Rosh Hashana is two days long, the second night, the night which starts the second day of Rosh Hashana, is still Rosh Hashana, and a good time for a family meal also (and it's customary to eat apples and honey at that time). Apples and honey can be eaten during the day, also: there's no harm in so doing AFAICT, and I've heard that some people do. Also, the blessings that should have been said the night before should be said by day (SA OC 271:8).