It says in SA OC Siman 472 that one should hurry to start the Seder in order that the children shouldn't fall asleep. If the person doesn't have any children is there still a reason to hurry to start the Seder?
In general, we attempt to fulfill mitzvos quickly (be-zrizus). This quality is patricularly praiseworthy at the time of Pesach, since the scriptural allusion to it is derived from the passuk "and you will guard the matzos" which in drush is read "you will guard the mitzvos" and the midrash derives - mitzvah that comes into your hand, "al tachmitzena" - don't let it become chametz - i.e. don't mix it with time, don't delay it.
Rav Moshe Shapiro explains based on the Maharal that mixing it with time destroys the spiritual effect of the mitzvah.
So Seder, as all mitzvos, must be gotten to quickly.
On a separate note, some people are strict with opinion that afikoman needs to be eaten before chatzos, and that leaves not a lot of time to go through the haggadah, so people try to start early.