DanF previously asked about piyutim and zemiros which follow the author's name. But if they don't follow the author's name, they follow the aleph-beis in some fashion or another (repeat letters, forwards, backwards, at-bash, etc.). Why is this? Why are there so few piyutim that are in an arbitrary order?
It is worth noting that according to Maharzu, Koheles Rabbah 1:33 says that although many paytanim attempted this pattern, they often were unable to come up with a stanza for all letters, while Shlomo HaMelech was always able to complete his piyutim - and have 5 extra, which Eitz Yosef understands as corresponding to the five final letters םןץף״ך. The Midrash based this off of Melachim Aleph 5:12. According to this reading of the Midrash, this practice of following the Aleph Beis is at least as old as Bayis Rishon, if not older.