The Gemara does not mention this side-effect. It just mentions the loss of "reward" for fighting the yetzer hara of avodah zara. There may have been other effects, and it may be that this reward includes specific spiritual benefits, such as the ability to reach Nevuah. For example, I think R.A. Kaplan says that the yetzer hara for avodah zara was the desire to reach pseudo-spiritual meditative highs without the hard work necessary to reach real ruach haKodesh. When people lost this 'meditative drive' they also lost the connected 'ability' for ruach haKodesh and Nevuah.
Sanhedrin 64a:
.."And they cried with a loud voice unto the Lord their
God." Now what did they say? — .. 'Woe, woe, it is that [sc. idolatry]
which destroyed the Sanctuary, burnt the Temple, slew the righteous,
and exiled Israel from their land; and still it sports amongst us!
Hast Thou not set it before us that we might be rewarded [for
withstanding its allurements]? But we desire neither temptation nor
reward!'... They fasted for three days, entreating for
mercy; thereafter their sentence fell from Heaven, the word emeth
[truth] written upon it... The shape of a fiery lion's
whelp issued from the Holy of Holies, and the Prophet said to Israel,
That is the Tempter of Idolatry. Whilst they held it fast, a hair [of
its body] fell out, and his roar of pain was heard for 400 parasangs.
[In perplexity] they cried: 'What shall we do? Maybe Heaven will pity
him !' The prophet answered: Cast him into a lead cauldron, and cover
it with lead to absorb his voice, as it is written, "And he said, This
is wickedness; and he cast it into the midst of the ephah: and he cast
the weight of lead upon the mouth thereof"..