Yochanan didn't leave Judaism; he became a Sadducee, meaning that he left the Pharisees. If we follow Abbaye's opinion that Yannay is Yochanan (Berachot 29a), then the story of how he became a Sadducee is given in Kiddushin 66a. Rava's opinion is that they are two separate people, but the story is still useful because it explains the motives why someone might convert from being a Pharisee to being a Sadducee. Here are the relevant parts of the story (translation modified from Sefaria):
היה שם זקן אחד ויהודה בן גדידיה שמו ויאמר יהודה בן גדידיה לינאי המלך ינאי המלך רב לך כתר מלכות הנח כתר כהונה לזרעו של אהרן שהיו אומרים אמו נשבית במודיעים ויבוקש הדבר ולא נמצא ויבדלו חכמי ישראל בזעם ויאמר אלעזר בן פועירה לינאי המלך ינאי המלך הדיוט שבישראל כך הוא דינו ואתה מלך וכהן גדול כך הוא דינך ומה אעשה אם אתה שומע לעצתי רומסם ותורה מה תהא עליה הרי כרוכה ומונחת בקרן זוית כל הרוצה ללמוד יבוא וילמוד
Now there was a certain elder present called Yehuda ben Gedidya, and Yehuda ben Gedidya said to King Yannai: King Yannai, the crown of the monarchy suffices for you. Leave the crown of the priesthood for the descendants of Aaron. As they would say that Yannai’s mother was taken captive in Modi’in. And the matter was investigated and was not discovered. And the Sages of Israel were expelled in the king’s rage.
And Elazar ben Po’ira said to King Yannai: King Yannai, such is the judgment of a common person in Israel. But you are a king and a High Priest. Is this your judgment as well? Yannai replied: And what should I do? Elazar responded: If you listen to my advice, crush them. Yannai countered: But what will become of the Torah? He retorted: Behold, it is wrapped and placed in the corner. Anyone who wishes to study can come and study. We have no need for the Sages.
In other words: a Pharisee insulted him by claiming that he wasn't a proper priest because his mother was raped, which led him to suspect the Pharisees of intending to rebel against him, and so he expelled the sages. Afterwards, El'azar ben Po'ira advised him that anyone who wishes can come and study the Torah. As Rav Nachman bar Yitschak adds here, his statement that whoever wants to learn the Torah can do so without the sages was a repudiation of the Oral Torah, meaning that he believed that the Written Torah was all that was necessary.
If we follow Abbaye in identifying Yannay with Yochanan, these would be the reasons that led him to become a Sadducee. If we follow Rava, then this isn't directly about the same person, but the motives (disputes with the Pharisees and the belief that the Torah could be learned without the traditions of the sages) could have been similar in his case.