Shabbos is exactly 24 hours according to the Torah (See maseches Moed koton daf 3b-4a where the Gemora tries to compare Shmitta to shabbos Which is Me-eis Le-eis Min Hatorah)and tosefes shabbos is a very short amount of time a matter of seconds (tos moed koton 4a), but we don't know exactly when it starts and ends so we keep from shekia (sunset) which is the start of bein hashmoshos (the unknown zone) and motzei shabbos ends at tzeis (either night fall or 72 minutes after sunset) that period we keep shabbos as a must. . and tosefos 4a "ma". The Ran in 84a yumo says that one can not possibly add to shabbos while the sun is out from the previous day. Indeed the Beier Halacha quoting Pri Megadim orach Chaim 261,2 says (following the vast majority of Poskim) before shekia (sunset) Tosefes Shabbos is miderabanan addition.
עיין בפמ"ג שכתב דמי שמקבל שבת בתוך הד' מילין הוא דאורייתא ובעשה דתוספת שבת ומן הזמן הזה עד פלג המנחה הוא רק מדרבנן:.
The Shulchan Aruch Yore Dea 87,3, Rema, Taz, Pri Chodosh and Creisi upleisi hold that Marit haayin does not apply to derabbanan cases.
Pri Chodosh:
הבו דלא להוסיף עלה
its enough so don't decree Maarit Haayin to rabbinic enactments except for the cases mentioned by Chazal
Shulchan Aruch:
ובשר חיה ועוף -- אפילו בחלב טהורה -- מותר בבישול ובהנאה, ואף באכילה אינו אסור אלא מדרבנן -
Chicken and kosher milk are permitted to be cooked together and derive benefit from even though the mixture is prohibited to eat, since it is only forbidden to eat Miderabbonon.
Rema:
הגה: ונהגו לעשות חלב משקדים ומניחים בה בשר עוף הואיל ואינו רק מדרבנן אבל בשר בהמה יש להניח אצל החלב שקדים משום מראית העין -
therefore we have the practice of cooking chicken with almond milk for us to eat since the prohibition of real milk cooked with chicken to eat is only derabanan. But animal (Beef lamb or Goat) meat one must put almonds next to the almond milk when cooked together because of Marit Haayin (i.e in case someone sees thinking that the meat was cooked with animal milk and permit himself). (See shachs* explanation of the Rema slightly differently)
But the Maharshal (quoted by Shach Yore Dea 87,3 who agrees and Noda Bihuda Tinyana O.C 30) says that Marit Haayin does apply to a Gezeiro Derabanan like eating(see Shach i quote below from 87,7) Chicken cooked with almond milk because people will think you are transgressing a Rabbinic Prohibition of chicken and milk. See Pri Chodosh in Yore Deah who defends Rema and Mechaber.
Despite the Maharshals objection the Ben Ish Chai says that everyone agrees in certain cases that there is no Maris Haayin.
Keeping Tosefet Shabbat is not Shabbos for everyone then we have a rule that we don't make a Gezeira of Marit haayin. In Sheilos Uteshuvos Rav Poalim from the Ben ish Chai http://hebrewbooks.org/pdfpager.aspx?req=1401&st=&pgnum=10 he talks about a push bicycle (In those days bicycles were without peddles and one pushed with his legs like a scooter) and says there is no Gezeira of Maris Haayin when riding a pushbike (on shabbos within an eiruv) that one might come to drive a car (Havara-combustion) or go on a carriage pulled by animals (Mechamer- making animal do a melacha), since there is a clear difference between what is ossur (car) and Mutar (push bike) and not get confused. So too here In Tosefes Shabbos in a large community where there are a lot of people that don't keep extra tosefes shabbos and it is known that it is optional and that those people are not breaking Tosefes Shabbos, so one will not get confused between what is Ossur (breaking Tosefes shabbos) and Muttar (accepting Shabbos later) when you Daven late and there is no Marit Haayin They know you are not doing a Melacha after accepting Shabbos rather you are davening later like so many others.
The Shach in Yore Dea also says clearly there is no marit Haayin when there is a evident possibility that you are doing something Muttar This is based on the Shach in Yore Dea 87,7 when describing cooking non kosher milk with meat-
בבישול לחודיה ליכא משום מראית העין דהא יכול להיות שמבשל לצורך רפואה או שאר דברים
The cooking of it (non kosher milk with meat or chicken with almond milk) is not prohibited because of Marit haayin (that you might eat it), since one can assume you are cookng(as opposed to eating where the Shach is quoted above that there is Marit haayin to eat chicken with almond milk) the mixture for an unwell person (in danger) or for another purpose e.g selling to a gentile (but when a healthy person eats chicken with almond milk there is no alternative but to think he is sinning).
Conclusion it is not marit haayin to work early one week and late the next as long as you don't do Melacha beyond 14 minutes before Shekia (CYLOR if you need more time).
*{ש"ך: ולא דמי לדם דגים סוף סימן ס"ו שמותר לגמרי ואפ"ה אם כנסו בכלי אסור מפני מראית עין אא"כ יש בו קשקשים שניכר שהוא דם דגים דהתם דם גופיה אסור מדאורייתא הלכך הרואה יאמר דאכיל דם בהמה שהוא אסור מן התורה אבל בשר עוף בחלב ממש הוא מדרבנן ליכא למיחש משום מראית עין Shach: Chicken cooked with milk Is not the same as Fish blood - which is permitted to consume together with the fish but once separated in a cup is forbidden because of Marit Haayin (i.e people will think your drinking animal blood which is forbidden Min haTorah) So add scales to the cup so people know its fish blood - Since there is no Marit haayin on a prohibition Derabanan.}