By Peter Apps, Political Risk Correspondent
WASHINGTON (Reuters)- For those who believe in a clash of civilizations between the Islamic world and Western democracy, the last few weeks must seem like final confirmation of their theory.
Even those who reject the term as loaded and simplistic speak sadly of a perhaps catastrophic failure of understanding between Americans in particular and many Muslims.
The outrage and violence over a crude film ridiculing the Prophet Mohammad points to a chasm between Western free speech and individualism and the sensitivities of some Muslims over what they see as a campaign of humiliation.
There seems no shortage of forces on both sides to fan the flames. The tumult over the video had not even subsided when a French magazine this week printed a new cartoon showing the prophet naked.
"It's ridiculous," Zainab Al-Suwaij, executive director of the America Islamic Congress, said of the violence that on Friday killed 15 in Pakistan alone as what were supposed to be peaceful protests turned violent.
"Yes, this video is offensive but it is clearly a grotesque over reaction that in part is being whipped up by radical Islamists in the region for their own ends. But it does show you the depth of misunderstanding between the cultures."
Starting last week with a few relatively small embassy protests and a militant attack in Libya that killed the U.S. ambassador and three others, violence has since spread to more than a dozen countries across the Middle East and Asia.
Despite the focus on religion, few doubt there are other drivers of confrontation.
The war on terrorism, U.S. drone strikes, wars in Iraq and Afghanistan and the Guantanamo Bay prison simply continue, in many Muslims' perceptions, centuries of Western meddling, hypocrisy and broken promises.
Meanwhile, many Americans see those regions as an inexplicable source of terrorism, hostage-taking, hatred and chaos. In Europe, those same concerns have become intertwined with other battles over immigration and multiculturalism.
"It has always been a difficult relationship and in the last decades it has become even more delicate," said Akbar Ahmed, chair of Islamic studies at American University in Washington. "Even a seemingly minor matter can upset the balance. ... What is needed is more sensitivity and understanding on both sides, but that is difficult to produce."
Not all the news from the region indicates an unbridgeable gap. Many Libyans, especially young ones, came out to mourn Ambassador Chris Stevens after his death and make clear that militants who killed him did not speak for them. Thousands of Libyans marched in Benghazi on Friday to protest the Islamist militias that Washington blames for the attack.
SPREADING DEMOCRACY AND MAKING FRIENDS
Still, the "Arab Spring" appears not to have made as many friends for America as Americans might have hoped.
The very countries in which Washington helped facilitate popular-backed regime change last year - Egypt, Tunisia, Libya and Yemen - are seeing some of the greatest anti-West backlash.
The young pro-democracy activists who leapt to the fore in 2011, Washington now believes, have relatively little clout. That leaves U.S. and European officials having to deal with groups such as the Muslim Brotherhood.
There is concern that regional governments such as Egypt might now be playing a "double game", saying one thing to the U.S. while indulging in more anti-Western rhetoric at home.
It may be something Washington must get used to.
"What you're seeing now is that (regional governments) are much more worried about their own domestic population - which means being seen as too close to the U.S. is suddenly ... a liability," says Jon Alterman, a former State Department official and now Middle East specialist at the Centre for Strategic and International Studies.
The current U.S. administration is not the first to discover democracy does not always directly translate into the sort of governments it would like to see.
In 2006, the election victory of Islamist group Hamas in the Gaza Strip was seen helping prompt the Bush White House to abandon a post-911 push towards for democratic change, sending it back towards Mubarak-type autocrats.
Rachel Kleinfeld, CEO and co-founder of the Truman National Security Project, a body often cited by the Obama campaign on foreign policy, said the new political leadership often had less flexibility than the dictators before them.
"Is that difficult for the U.S.? Yes, of course. But it would be a mistake to simply look at what is happening and decide we should go back to supporting autocrats," she said.
The popular image of the United States in the Middle East stands in stark contrast to the way Americans view themselves.
Western talk of democracy and human rights is often seen hollow, with Washington and Europe only abandoning autocratic leaders when their fate was already sealed and continuing to back governments such as Bahrain still accused of repression.
"The simple truth is that the American people are never going to understand the region because they never ask the right question - which is what it feels like to be on the receiving end of American power," says Rosemary Hollis, a professor of Middle Eastern studies at London's City University.
Whoever wins the White House in November will face a string of challenges across the region.
As it faces down Iran over its nuclear program, while backing rebels in Syria and governments in the Gulf, Washington risks being drawn ever deeper into the historic Sunni-Shi'ite sectarian divide within Islam.
Already having to face up to its dwindling influence over Iraq, it must broker its exit from Afghanistan and try to keep nuclear armed Pakistan from chaos.
Then, there are relations with its two key regional allies, Israel and Saudi Arabia, both troublesome in different ways.
Israel is threatening military action against Iran over its nuclear program, and U.S. officials fear Americans would feel the consequences if Israel does attack.
The Israeli-Palestinian conflict remains deadlocked, and Obama's rival for the presidency, Republican Mitt Romney, indicated in comments earlier this year and made public this month that he sees little chance of any change there.
Saudi Arabia might be a key oil producer and occasionally invaluable ally, but analysts say some rich Saudis, if not the government itself, have long funded and fueled Islamist and Salifist extremism and perhaps also Sunni-Shi'ite tension.
Said Sadek, professor of politics at the American University in Cairo, said people in the Middle East still prefer Obama to the alternative. "He is seen as the only president to ever really reach out to the Middle East. But (it) is a difficult place," he said. "The countries that have gone through revolutions were always going to be unstable. ... You could have perhaps 5 to 15 years of instability."
While many Americans would like nothing more than to turn their backs on the region, Obama made clear this week he does not see that as an option: "The one thing we can't do is withdraw from the region," he said. "The United States continues to be the one indispensable nation."
(Editing by Warren Strobel and Claudia Parsons; Desking by Jackie Frank)