Parsha Shelach 13:33 says the following:

And there we saw the Nephilim, the sons of Anak, who come of the Nephilim; and we were in our own sight as grasshoppers, and so we were in their sight.

The Lubavitcher Rebbe ellaborate on this (see Likkutei Sichos vol. 28, p. 89) and says that these Nephilim were the fallen ones, i.e. malachim that fell in the days of Enosh and they were also involved with the origins of idolatry. How did this happen? Did they originated the idea and gave it to the people or, being only malachim, did they were affected by the evil of that generation?