Rabbi Meir is not a good example of whether it is good or not to learn from an apostate. Although he was one of the most learned of the Tannaim, he made many tragically bad decisions in his life. Rashi brings down a tradition, in his comment to Avodah Zara 18b, that Rabbi Meir arranged the seduction of his wife by one of his pupils in order to prove the validity of the Talmudic claim that women are "light-minded." After many refusals, Beruriah finally yielded to the student's sexual advances. When she realized that her husband had set the trap for her, she hanged herself and Rabbi Meir ran away out of shame into self-imposed exile.
In another story, told at Horayos 13b to 14a, Rabbi Shimon ben Gamliel (Rebbe's grandfather) was the Nasi and Rabbi Noson was the Av Bais Din. Rabbi Meir plotted to depose Rabbi Shimon by suggesting that Rabbi Noson call on him to teach Uktzein, which he knew Rabbi Shimon had never learned. If Rabbi Shimon was effectively sandbagged, then he would be shamed and have to step down as Nasi, to be replaced by Rabbi Noson, and allowing Rabbi Meir to become the Av Bais Din. Rabbi Yaakov ben Kodshi got wind of the plot and warned Rabbi Shimon, who crammed during the night and successfully gave the lecture the next day. Rabbi Shimon expelled Rabbi Noson and Rabbi Meir from the bais midrash and instructed that any questions that those inside could not answer would be taken to the two rabbis outside. Rabbi Shimon eventually let them back in, but decreed that they be fined and that after that date, none of their subsequent teachings could be quoted in their own names, but would Rabbi Meir's statements would be cited to "Some say" and Rabbi Nosson would be cited as "Others say." Rebbe (Rabbi Yehuda HaNasi) followed this rule when he established the Mishnah.
I cannot speculate whether these shocking stories were a product of Rabbi Meir's association with Acher (Rabbi Elisha ben Avuyah), but it is clear from the story you cite in Chagigah 15b that Acher had lost his own moral compass already. There, the story is told that Rabbi Meir tried to bring Acher back to the beis midrash even though Acher stated that he had "heard from the other side" that his repentence would not be accepted. They come across a young boy and Acher either (a) butchered the boy or else (b) said that he would have killed the boy if he had a knife.
All in all, I don't think the relationship of Rabbi Meir and Acher is an appropriate case from which to learn that it is permissible to study with an apostate.