The Rambam apparently believes that the tinoq shenishba is defined by the attitude he/she was raised with toward Torah, not only their ignorance of it. And so even after learning Torah, as long as his motivation was at least founded on the initial bias against believing its truth or applicability, the tinoq shenishba is not held to account.
To quote the Rambam's Code, Hilkhos Mamrim 3:1-3 (emphasis mine, tr. R' Zvi Lampel):
בַּמֶּה דְּבָרִים אֲמוּרִים בְּאִישׁ שֶׁכָּפַר בַּתּוֹרָה שֶׁבְּעַל פֶּה בְּמַחֲשַׁבְתּוֹ וּבִדְבָרִים שֶׁנִּרְאוּ לוֹ. וְהָלַךְ אַחַר דַּעְתּוֹ הַקַּלָּה וְאַחַר שְׁרִירוּת לִבּוֹ וְכוֹפֵר בַּתּוֹרָה שֶׁבְּעַל פֶּה תְּחִלָּה כְּצָדוֹק וּבַיְתּוֹס וְכֵן כָּל הַתּוֹעִים אַחֲרָיו. אֲבָל בְּנֵי הַתּוֹעִים הָאֵלֶּה וּבְנֵי בְּנֵיהֶם שֶׁהִדִּיחוּ אוֹתָם אֲבוֹתָם וְנוֹלְדוּ בֵּין הַקָּרָאִים וְגִדְּלוּ אוֹתָם עַל דַּעְתָּם. הֲרֵי הוּא כְּתִינוֹק שֶׁנִּשְׁבָּה בֵּינֵיהֶם וְגִדְּלוּהוּ וְאֵינוֹ זָרִיז לֶאֱחֹז בְּדַרְכֵי הַמִּצְוֹת שֶׁהֲרֵי הוּא כְּאָנוּס וְאַף עַל פִּי שֶׁשָּׁמַע אַחַר כָּךְ [שֶׁהוּא יְהוּדִי וְרָאָה הַיְהוּדִים וְדָתָם הֲרֵי הוּא כְּאָנוּס שֶׁהֲרֵי גִּדְּלוּהוּ עַל טָעוּתָם] כָּךְ אֵלּוּ שֶׁאָמַרְנוּ הָאוֹחֲזִים בְּדַרְכֵי אֲבוֹתָם הַקָּרָאִים שֶׁטָּעוּ. לְפִיכָךְ רָאוּי לְהַחְזִירָן בִּתְשׁוּבָה וּלְמָשְׁכָם בְּדִבְרֵי שָׁלוֹם עַד שֶׁיַּחְזְרוּ לְאֵיתָן הַתּוֹרָה:
.... However, this is speaking solely of those who ... initiate such denial, such as Tzadoq and Boethus. But the children and grandchildren of these kinds of men, who were misled by their fathers, that is, those who were born into and bred by the Qaraites, are like people who while still young had been taken into captivity and raised in another religion. Such captives are not quick to take hold of the mitzvos, for their situation is practically beyond their control: Even if they later hear that they are Jewish and are exposed to Jews and Judaism, they still are considered people who practice another religion only by force, since they were raised and educated erroneously. The same is true with those who keep the ways of their erroneous Karaite fathers.