You are probably miscalculating the actual age of Noach at the time that Shem was born. Calculate using only the ages given in Noach starting from when Shem fathered Arpachshad. Then add the time to the date of the flood. The timeline in the Art Scroll Chumash (as one example) puts the date of the flood as 1656 AM. I did the timeline myself based on the pesukim and did indeed show Noach was born in 1056 and the flood was in 1656 (as posted in Rabbi Leibtag shiurim: The Hebrew Calendar and its Missing Years- Parts 1 - 3
Noach 11:10
These are the generations of Shem: Shem was one hundred years old, and
he begot Arpachshad, two years after the Flood.
If you start from here and add the ages at the birth of each generation, you get (using the 70 years of Terach)
Noach 11:26
And Terah lived seventy years, and he begot Abram, Nahor, and Haran.
a total of 292 years after the flood. This now give 1656 + 292 = 1948.
This assumes that Avraham was the eldest and was born when Terach was 70, unlike Shem who was the definitely born after Yafes (two years after Noach started having children at 500), as shown in the pasuk.
Noach 10:21
And to Shem were also born [children; he was] the father of all the
people of the other side [of the river], the brother of Japheth the
elder.
Rashi
the brother of Japheth, the elder: I do not know [from the wording of the verse] whether Japheth is the elder or Shem. [The verse may be
rendered: the elder brother of Japheth.] When Scripture says (11:10):
“Shem was a hundred years old, etc.” two years after the Flood, you
must say that Japheth was the elder, because Noah was five hundred
years old when he first had children, and the Flood came to pass in
the six hundredth year. Consequently, the eldest of his sons was a
hundred years old [at the time of the Flood], and Shem did not reach
[the age of] a hundred until two years after the Flood. — [from Gen.
Rabbah 37:7]
Rashi says Cham was not the youngest, though Rav Hirsch translates Noach 9:24 as his youngest son
. However, this does not change the timeline based on 11:10.
And Noah awoke from his wine, and he knew what his small son had done
to him.
Rav Hirsch translates this pasuk as:
When Noach awoke from his wine and knew what his youngest son had done
unto him.
Rashi
his small son: The worthless and despicable one, like (Jer. 49:15): “Behold I have made you small among the nations; despised
among men.” - [from Gen. Rabbah 16:7]