Shabbat 147b features one version of a certain story about Rabbi Elazar ben Arakh. In this version, he forgets all of the Torah he learned because he was drawn to places with quality wine (Perugaita) and hot water baths (Deyomset):
"The Gemara relates that once Rabbi Elazar ben Arakh happened to come there, to Perugaita1 and Deyomset,2 and he was drawn after them, and his Torah learning was forgotten. When he returned, he stood to read from a Torah scroll and was supposed to read the verse: “This month shall be for you [haḥodesh hazeh lakhem]” (Exodus 12:2), but he had forgotten so much that he could barely remember how to read the Hebrew letters, and instead he read: Have their hearts become deaf [haḥeresh haya libbam], interchanging the similar letters reish for dalet, yod for zayin, and beit for khaf. The Sages prayed and asked for God to have mercy on him, and his learning was restored."
Another version appears in Kohelet Rabbah 7:7, where Rabbi Elazar follows his wife to Emmaus. None of his friends come with him and he eventually comes to forget basic laws:
"Rabbi Yoḥanan ben Zakkai had five disciples. As long as he was alive, they would sit before him. When he died, they went to Yavne, but Rabbi Elazar ben Arakh went to his wife in Emmaus, a place with good water [/beautiful waters] and a beautiful view. He waited for them to come to him, but they did not come. When they did not come, he sought to go to them, but his wife did not allow him to do so. She said: ‘Who is in need of whom?’ He said to her: ‘They are in need of me.’ She said to him: ‘A leather container [of food] and mice, which typically goes to which; the mice to the container, or the container to the mice?’ He heeded her and remained until he forgot his learning. Sometime later they came to him and asked him: ‘A wheat loaf or a barley loaf, which can be eaten faster with a relish?’ He did not know how to respond to them, or the meaning of “with relish [liftan].” Rabbi Eliezer and Rabbi Yosei say: It is two food items joined [lefutin] together.
It should be noted that two other versions of this story appear in Avot D'Rabbi Natan version A and B (see here) but neither specifically say that he forgot his Torah learning. They just say that he did not become great in the field of Torah because he went to Dimsit2/Emmaus because of their 'beautiful waters'. It has been suggested by Judah Goldin in his Hebrew essay "משהו על בית מדרשו של רבן יוחנן בן זכאי" (Something About the Study Hall of Rabban Yochanan ben Zakkai), The Tzvi Wolfson Jubilee Book, pp. 79-80 that these 'beautiful waters' symbolize study material that wasn't considered entirely acceptable in rabbinic circles, i.e., Hellenistic philosophical studies.3
These stories contain a lot of symbolism and a number of essays have discussed them. One particularly key point is, with regards to your question, that the Torah has also been compared to water (see for example Bava Kamma 82a, Taanit 7a, Beresheet Rabbah 41:9), so it seems that going after other sorts of 'watery' pleasures (whether physical or intellectual) brings about loss of truly important 'water' - Torah.
1 There are different views as to which place this name refers to. Sefaria translated it as Phrygia, but there are equally plausible suggestions that it refers to somewhere in the Galilee or in Transjordan.
2 Though in Shabbat 147b and Avot D'Rabbi Natan version A Deyomset/Dimsit is presented as a place-name, the word literally means 'hot water bath'.
3 Goldin also published an English version of this essay: J. Goldin, 'A Philosophical Session in a Tannaite Academy', Traditio 21 (1965), pp. 1-21 (JSTOR link) but I was not able to find this particular idea mentioned there, though it's possible I missed it. The Hebrew essay is not available online to my knowledge, but I can provide a scan if it interests you.