The Maharal on Avos suggests an answer that combines two of the already given ideas.
Hillel and Shammai themselves only disagreed three or maybe five times. Shabbos 15a, Chagiga 2:2, Eduyos 1:2-3, Niddah 1:1) Then their students have 316 recorded disputes! The explanation for this explosion in number is given as "they did not serve their mentors" (Sanhedrin 88b), a lack of proper apprenticeship.
The Maharal points out that Hillel and Shammai were the last in a series of zugos, "pairs" of Nasi (Prince / President) and As Beis Din (Head of the High Court) who together comprised the rabbinic leadership. A sequence of mishnayos that make up part of Avos ch. 1 are sets of quotes from each of the zugos. The Maharal shows how in each generation, the nasi is quoted a giving advice based on chesed, lovingkindness, whereas the av beis din makes a somewhat parallel statement but one coming from din, strict justice. Including Hillel and Shammai. Which fits each role: the nasi's job is to distribute charity and otherwise provide for the communty, whereas the av beis din is the head of the justice system.
He explains that without the hands-on service of their mentors, the students confused their mentor's jobs with their positions. The Students of Hillel received an image of Hillel's Torah that was overly influenced by his happening to be nasi, and therefore chessed dominated. Similarly, Shammai's students and din.
As for why we follow Beis Hillel, there was a bas qol, an echo of a metaphysical voice, that told us to. But according to Tosafos and general consensus since, we only follow this bas qol because it reaffirmed the normal rules of establishing halakhah -- majority rules, even when the minority is brighter. Beis Hillel were greater in number, but Beis Shammai were sharper. (As per the gemara that mentions that bas qol.) See my answer at https://judaism.stackexchange.com/a/67460/1570