Maadanei Yom Tov says that we are counting days since the day commemorating Yetzi'as Mitzrayim. So that "hayom yom echad ba'omer" means one day elapsed since that moment in time, not that it is the first day. We are indeed counting "at the end" of one day, as you put it. Here is his argument:
Why is it that we celebrate Shuvu'os on the 6th of Sivan, and call it Zeman Matan Toraseinu -- the time of the giving of our Torah, when the Torah was actually given on the 7th?
Also, why is the Torah ambiguous about how many days to count? Quoting Vayiqra 23:15-16:
וּסְפַרְתֶּ֤ם לָכֶם֙ מִמָּחֳרַ֣ת הַשַּׁבָּ֔ת מִיּוֹם֙ הֲבִ֣יאֲכֶ֔ם אֶת־עֹ֖מֶר הַתְּנוּפָ֑ה שֶׁ֥בַע שַׁבָּת֖וֹת תְּמִימֹ֥ת תִּהְיֶֽינָה׃
עַ֣ד מִֽמָּחֳרַ֤ת הַשַּׁבָּת֙ הַשְּׁבִיעִ֔ת תִּסְפְּר֖וּ חֲמִשִּׁ֣ים י֑וֹם וְהִקְרַבְתֶּ֛ם מִנְחָ֥ה חֲדָשָׁ֖ה לַה׃
And from the day on which you bring the sheaf of elevation offering — the day after the day of rest — you shall count off seven weeks; they must be complete.
You must count until the day after the seventh week — fifty days; then you shall bring an offering of new grain to Hashem.
Are we counting 7 weeks = 49 days, or 50 days? And while one might be tempted to say that Shavuos "the day after the seventh week" is the 50th day, what the verse actually says is to count 50 days until the day after the seventh week -- the weeks we were just talking about when we said 49. Not including it.
The answer is that we're counting from day 0. For us today, day 0 is the first day of Pesach. On the second day of Pesach we say "This is one day in/toward the omer" -- it's a day since day 0. And we would be bringing the qorban omer to mark the completion of 1 day, thus we count the time completed.
However, on the original Pesach, day 0 didn't begin at nightfall. The actual Exodus was during the day. And since (as per the quoted verse) omer has to count complete days and weeks, that meant their day 0 was on the second day of Pesach. They counted 49 days from that time, and thereby reached the 7th, not the 6th of Sivan.
So, we count a duration of 49 days, but because there is a 0th day, the last day of our count is a 50th day. Thus, both verses are fulfilled.
But, that means we are counting days from a t=0, and thus days too are "counted at the end" of the day.