Since you're interested in the "secular" explanations for anti-Semitism, I suggest you learn about the history of anti-Semitism. There are countless good books on this subject.
I am not a historian, but I am familiar with some of the historical literature on this topic. So I will give you a brief explanation of my understanding of why the Nazis hated and tried to kill all Jews, regardless of their level of observance.
Traditional Christian anti-Semitism was brutal, and often resulted in not only oppression but also periodic massacres of Jews. Martin Luther, the founder of Protestantism, even suggesting rounding all Jews into camps and killing them (in a book called The Jews and their Lies), but no Christians ever went that far. The Nazis had a completely different ideology than these Christians, which made them much worse.
Nazism rejected basic tenets of Judeo-Christian philosophy, in which each person is valuable in the eyes of G-d and deserves and basic respect and dignity. They were inspired by the philosopher Nietzsche, who criticized Judeo-Christian morality as a "slave morality" rather than the supposedly noble morality of the "superman," and by the ultra-nationalist and pseudo-scientific notion that Germans were the master race and destined the rule over inferior peoples.
They also had the strange and contradictory belief that the Jews were the root of all evil. Whatever problems existed in society--the excesses of capitalism, revolutionary socialist movements, prostitution, cosmopolitanism--they blamed it on the Jews. They believed that whether or not the Jews were religious, there was some aspect to their soul or biology that caused them to bring evil to the world. Jews were in fact over-represented among bankers, stock-brokers, tax collectors, intellectuals and socialist activists, but of course this doesn't prove that Jews were somehow driven to spread evil.
Many Germans were probably also inspired by the Tzarist forgery the Protocols of the Elders of Zion, which claimed that Jews were conspiring to run the world to their benefit and everyone's else's detriment. This document remains popular around the world, particularly the Muslim world, and the Iranian government even prints its own official copies. Of course the Protocols are completely untrue, but they are a significant influence on anti-Semitism throughout the world.
Ultimately, the differing cultural characteristics of the Jews, whether you're talking about observances or personality characteristics or whatever, that existed in reality had virtually nothing to do with why the Nazis did what they did. A bizarre and extreme ideology, something called eliminationist anti-Semitism (see the book Hitler's Willing Executioners), was the cause of the Holocaust. Mere resentment against Jews for being different culturally may have played some role in stoking pre-Nazi anti-Semitism, but it was not a significant factor in Nazi anti-Semitism.
As mentioned in another answer, Jews have traditionally believed that whatever bad things happened to us are sent by G-d in reaction to sin. However, this is a major area of debate, and it doesn't really answer your question, which is about secular explanations. But I'll just mention that not everyone believes the Holocaust was caused by G-d's reaction to many Jews' abandonment of the commandments. For example, the Lubavitcher Rebbe said that the sins of the Jewish people alone would not be enough to justify the Holocaust, even if all the sins from the beginning of time were added together.