Right. Unless you do some maneuvering, the top challah would be the closest, and mitzvas are first-come-first-serve, so all times other than Friday night, you make the bracha on the top one.
As for Friday night: we focus on Shabbos vis-a-vis the creation of the world. Then the world was there; but it's passive, serving as a basis for history. Hence, bottom challah.
Shabbos day we focus on Shabbos vis-a-vis the Exodus, and generally G-d's involvement in the world, that's active (riding on top of nature, so to speak). Top challah.
(From what I've heard, they correspond to the kabbalistic notions of Atika Kadisha and Chakal Tapuchin Kadishin, respectively, as referenced in their Askinu Seudasa songs.)