אבא שאול אומר הכונס את יבמתו לשם נוי ולשום אישות ולשום דבר אחר כאילו פוגע בערוה וקרוב אני בעיני להיות הולד ממזר
Not l'shem mitzvah means doing Yibbum for the beauty of the woman, for marriage, or for "something else" (which could either literally mean any other reason, or could be a euphemism for intimate relations). This is presumably a list of examples of anything other than the mitzvah.
Yibbum is a situation in which a very serious prohibition is overridden/suspended for the Mitzvah being performed. Because this situation is prohibited sans the mitzvah, Abba Shaul felt that one who is not focusing on the mitzvah is as if he is coming into contact with that prohibition. (According to Rashi's comments, it seems as if Abba Shaul held it was literally violating this prohibition - Rashi to Bechoros 13a s.v. עכשיו). Therefore, his intentions make a difference in this case, as it becomes a "mental" אשת אח when he does not intend for the Mitzvah.
R' Elchonon Wasserman in Kovetz Ha'aros 36 posits that when it comes to any positive commandment overriding a negative commandment not just Yibbum e.g Shaatnez on tzitzis, metzora or nazir cutting the corners by shaving their heads, even the opinion that normally holds Mitzvos Ein Tzrichus kavana (do not need proper intention) will agree that in this situation it is required for the positive commandment to override the negative completely and for their to be no repercussions having done the Aveira at the same time as the Mitzva.
If you follow this to its logical conclusion, this would mean that if someone went ahead and did Yibbum for the wrong reasons, the Yibbum would not take effect as a full Mitzva, as it is in violation at the same time of the prohibition of marrying a brother's wife, in which case marriage bears the iniquity of the Aveira even though it works,
The Chazon Ish (Nashim 129:13) understands that Abba Shaul and the Chachomim both understood that on a Biblical level this Yibbum works, and that even the Chachomim agree that ideally you need to have the proper intentions for the Mitzvah. Their argument was that the Chachomim understood this Mitzvah to be no different than any other Mitzvah, in which doing it "shelo lishma" is better than not doing it i.e the greater good of the mitzva supercedes any iniquity of the aveira even if he did it for the wrong intentions, whereas Abba Shaul held that in this particular Mitzvah you are violating the intent of the Torah, and from Heaven's perspective it is as if you have gotten involved with an illicit relationship, and it is therefore better to not do it even though a mitzva is being performed at the same time and the Marriage would fully take effect.
The phrasing of the halacha seems to be a blanket rule, not dependent on individual basis. In fact, when the Gemara in Yevamos suggests that the halacha reverted to be that Yibbum takes precedence, Rav Nachman exclaims in surprise "Has the generation improved?" and the response is that the consensus of who's opinion to follow has changed - the implication is that short of a change in the halachic conclusion, it is untenable that we would assume we have reached the point of doing Yibbum with the proper intentions.