Why is the first chapter break in B'reishit where it is, and not a few verses later at the beginning of the second creation telling (where we place the aliya break)?
Many chumashim at the end of each of the five books list the number of pesukim and the number of parshas and the number of chapters in each book. For example, at the end of the book of Bereishis it says that there are 1534 pesukim, twelve parshas and fifty chapters.
But note also that it says וסדריו מ"ג - and the number of sedarim is 43. These sedarim are the original, Jewish divisions of the Tanach handed down to us by the Baalei Hamesorah, and if you look in the Koren Tanach you will see these marked in the wider, outside margin (to give them more prominence). And the second sedra starts at Bereishis 2:4, which is the logical place as was pointed out in the question.
What follows is a summary of the historical background of this topic from the sefer מודע לבינה here, some of which has already been mentioned in previous answers and comments, but he adds more details and brings sources:
R. Eliyahu HaLevi in his introduction to his sefer הבחור writes that the division into chapters is not part of our tradition from the Baalei Hamesorah (the Masters of the Tradition), but are the invention of the person who first translated the Tanach into Latin (according to the Koren Tanach this was an English priest in the 13th century) and saw the need to divide up the books, and so divided them up according to his own logic. Later, in the year 5198 (1458) when R. Yitzchok Nasan came to write his sefer מאיר נתיב which is known as the Concordance, he was forced to use this division since it was already in widespread use in his day, and also because he did not have at that time an alternative division - that of the Baalei Hamesorah.
Later still, when printing presses became available and Sifrei Kodesh started to be printed, they also used this chapter division, and R. Yaakov ben Chaim in his introduction to the first edition of the Mikros Gedolos in the year 5286 (1546) apologized for this, saying that since he made extensive use of the Concordance in his editing of the Mikros Gedolos, he had to use the same divisions as that sefer had used. But he added that if he would have had available the division of the Baalei Hamesorah he would have preferred to use that. And afterwards, when he had nearly finished the preparations for printing, he finally found the division of the Baalei Hamesorah, and he decided to add it to the printing so that it would not become lost forever.
(In addition, the books of Shmuel, Melachim and Divrei Hayamim (Samuel, Kings and Chronicles are not divided into two according our tradition - this was also done by the Christians according to the Koren Tanach) _
As l' said, the chapter divisions are indeed of Christian origin. This article says that the originator of this division separated Shabbos from the other weekdays for reasons having to do with Christian theology, but doesn't specify how. I seem to recall reading a suggestion that the idea behind it was to downplay our Shabbos in favor of their Sunday.
According to Wikipedia, the Chapters were divided by Christians, based on their understanding of the bible. Although we use their system, we only use it for reference. Regarding understanding, we have our own division, comprised of setumot, petuchot, aliyot (to some extent), and parshiyot. The breakup of chapters in no way represents Jewish understanding.
Perhaps, the reason is to distinguish the holiness of Shabbat from the rest of the week, which, in comparison, is more mundane. Shabbat, being holy, is level two; and the rest of the week, being mundane, is only on level one.