There’s a way to beat Islamic State. But it won’t be pretty.

The Iraqi city of Ramadi is the last place more than 200 American soldiers and Marines drew breath, or where they suffered the wounds that would ultimately kill them. For a time, it was one of the most dangerous places in the country, the heart of the Sunni insurgency against America’s military occupation. It was here that al-Qaeda in Iraq, the terror franchise led by Abu Musab al-Zarqawi, once thrived.

But it was here, also, that Sahwa al-Anbar, or the Anbar Awakening, took hold in 2006. Outraged by al-Qaeda’s atrocities, tribal sheiks in the city turned against it, forming an unlikely alliance with the Americans their young men had previously fought. Together, the Americans and Iraqis expelled al-Qaeda from Ramadi.

The Awakening spread, its successes solidified by a “surge” of U.S. troops into Iraq the following year. What seemed to be an unstoppable spiral into violence in Anbar province was halted. More than 1,300 Americans died fighting in the province, but peace, a version of it anyway, seemed within reach.

Ramadi, then, represents the low and high points of the American invasion and occupation: loss and reconciliation, sacrifice and redemption. Those who fought there, American and Iraqi, could feel they had achieved something.

This May, the city was overrun by the so-called Islamic State, a militant group that grew out of al-Qaeda in Iraq and has slaughtered and raped its way across much of Iraq and Syria, declaring a caliphate in the territory it controls. Islamic State’s conquest of Ramadi came almost a year after it quickly and seemingly effortlessly seized Iraq’s second-most populous city, Mosul. It came nine months after U.S. President Barack Obama vowed to “degrade and ultimately destroy” it—launching an air campaign and training mission that would be joined by some 20 Western and Arab nations, including Canada.

Related: Is the fall of Ramadi a chance to bridge Iraq’s sectarian divide?

Yet, within days of conquering Ramadi, Islamic State, also known as ISIS and ISIL, took the Syrian city of Palmyra, home to some of the most impressive Roman-era ruins in the world. Militants reportedly forced civilians to watch them murder 20 supposed government supporters in the ancient amphitheatre. Another nearly 300 pro-government troops were reportedly executed, and photos from the city show decapitated bodies in civilian clothes lying in the streets.

Obama, who once likened Islamic State to a “JV,” or junior varsity basketball team wearing a professional team’s uniforms, described the fall of Ramadi as a “tactical setback.” America last week announced it would send to Iraq another 450 troops, who will work out of a new base in Anbar. But the White House has given no indication it is considering a change in its strategy of air strikes and training without deploying American combat troops.

In February, a U.S. official at Central Command told reporters that Iraqi and Kurdish fighters would launch an offensive to retake Mosul in April or May. That’s now off the table. An attack to retake Ramadi does not appear imminent, either.

“It’s not working. It’s a failed strategy,” says Feisal al-Istrabadi, a former Iraqi ambassador to the United Nations, speaking of the American-led campaign. “The idea that [the fall of Ramadi] is some kind of minor setback is a mistake. It’s much more serious than that. It shows that ISIL still has the ability to take the initiative on the ground. ISIL has a strategy. It’s metastasizing. It’s growing. It’s capturing more territory.”

---

Michael Pregent, a former U.S. intelligence officer who was embedded with Iraqi security forces, says American strategy against Islamic State is “in tatters”—in part because America’s reluctance to more fully engage in Iraq, in addition to giving Islamic State room to expand, has allowed Iran and Iran-backed militias to strengthen their influence over Iraq. The American air force, he says, is now providing air support to the same Iranian proxies that killed American soldiers during the U.S. occupation of the country and that seek to dominate Iraq themselves.

It’s a frustration shared by Michael Knights, who spent years in Iraq running a security company’s intelligence unit and is now a Lafer fellow at the Washington Institute for Near East Policy. “Our slow-burn security assistance, our drip-feed approach, means there is no other effective armed force in Iraq right now, other than the Popular Mobilization Forces,” he says, referring to the mostly Shia militias. They are officially under the command of the Iraqi government, but, in practice, Iran wields enormous control over them. “We are essentially serving up Iraq on a platter to these people,” says Knights.

Related: Time for Arab nations to step up in the fight against Islamic State

Obama has insisted that Iraq must lead the fight against Islamic State. America, he says, can help. But Iraqis, who remember how forcefully America overthrew the regime of Saddam Hussein 12 years ago, are angered by the slow pace of its campaign against Islamic State. “We see convoys going through cities in Iraq. Where are the satellites? Why aren’t [coalition planes] hitting them?” says George Mansour, a former minister in Iraq’s Kurdistan Regional Government who now lives in Canada. “I think the world is not serious about fighting ISIS, especially the United States.”

Obama’s desire to avoid more deeply entangling America in Middle Eastern wars has come at a cost. Syria, already nearly broken by dictatorship and civil war, has been further traumatized by Islamic State’s barbarism. Iraq has lost roughly a third of its territory to Islamic State, and more and more of its sovereignty is slipping into Iranian hands.

If Obama is genuinely committed to Islamic State’s destruction, America’s strategy needs to change. One option is for America to co-operate with Iran and its proxies and allies, including Syrian dictator Bashar al-Assad. The second is to escalate its own military intervention, possibly including a limited number of troops in combat roles. Obama shouldn’t consider the first option. The evidence so far suggests he won’t consider the second.

---

 

What are the origins of Iraq’s current nightmare? How did Islamic State recover in such strength after its al-Qaeda forebear had been nearly defeated? Syria’s civil war gave it new life: territory in which to grow, and a galvanizing cause with which to recruit new members. But the group also found fertile ground in Iraq, and some of the reasons for that can be traced to Iraq’s 2010 legislative elections.

The incumbent prime minister was Nouri al-Maliki, a Shia chauvinist who had spent years during Saddam’s dictatorship in Iran and Syria. His State of Law coalition finished second, with two fewer seats than the Iraqi National Movement, or al-Iraqiya, a secular nationalist coalition headed by Ayad Allawi that contained both Sunni and Shia leaders.

Maliki, however, refused to concede that Iraqiya should get the first chance to form a government and stayed on as prime minister. A deadlock lasting months ensued. “The formation of the government was perceived as a battle between Iran and the U.S.,” writes Emma Sky, a British former political adviser to top American general Ray Odierno, in her memoir, The Unraveling: High Hopes and Missed Opportunities in Iraq. “Everyone realized this, except for the Americans.”

U.S. Vice-President Joe Biden told Maliki America would support him, and urged Allawi to accept Maliki as prime minister, according to Sky. Biden, she says, did not understand that many Iraqis wanted to move past sectarianism. “Look, I know these people,” he told her. “My grandfather was Irish and hated the British. It’s like in the Balkans. They all grow up hating each other.” Biden later made the same sweeping statement to a baffled, multi-ethnic gathering of Iraqiya members.

Obama might have known better, had he paid attention, writes Sky, “but his only interest in Iraq was in ending the war.”

Maliki’s second term as prime minister saw the intensification of anti-Sunni discrimination and the steady growth of Iranian influence. Maliki reneged on a promise to integrate fighters from the Anbar Awakening into Iraq’s regular security forces and civil service, dismantling a force that had helped to free Iraq from al-Qaeda. He consolidated power in his own hands and purged rivals and Sunni leaders.

Iraq’s Sunnis felt they had few friends in their own government. This does not mean support for Islamic State runs deep among Iraq’s Sunnis. “You can see it in the flow of refugees,” says Istrabadi, who is now founding director of the Center for the Study of the Middle East at Indiana University. “They’re fleeing [from Islamic State] in every direction they can.”

But when Islamic State stormed through western Iraq, it confronted a weak and disunited country. Iraqi security forces melted before it. Former fighters from the Anbar Awakening had been disarmed and, in any case, felt little loyalty to Maliki’s government in Baghdad.

“When [U.S. Secretary of Defence] Ash Carter says they don’t have a will to fight, there’s some truth to that,” says Renad Mansour, a non-resident scholar at the Carnegie Endowment for International Peace’s Middle East Center (and George Mansour’s son). “But it’s also a bit problematic, because there is a will to fight. The problem is that the lack of trust stops that will.”

“What exactly would a Sunni tribal sheik in Anbar tell his followers they are fighting for in Anbar?” asks Istrabadi. “I know what you can say they would be fighting against: ISIL. But what is he telling them they are fighting for?”

With Iraq’s official security forces collapsing, last June, Ali al-Sistani, Iraq’s leading Shia cleric, issued a fatwa, or religious edict, calling on Iraqis to defend their country. This was the beginning of the Popular Mobilization Forces, known as al-Hashd al-Shaabi. They incorporated previously existing militias, such as the Badr Brigades, many of which are backed by Iran. Qassem Soleimani, commander of Iran’s Revolutionary Guards’ Quds Force, is often seen among them.

Knights says Iran is intent on “colonizing” Iraq through militias and political parties it supports. Iran, like Iraq, suffered horribly during the 1980-88 war between the two countries (launched when Iraq invaded Iran). Tehran wants to ensure Iraq can never again threaten it, says Knights.

Despite Iranian support for the Popular Mobilization Forces, Istrabadi says those who joined them fight for Baghdad rather than Tehran. “Iraq’s Shia bled for eight years in the war with Iran. What more do they have to do to prove their loyalty to Iraq? Eight years wasn’t enough?” he says. “It so happens that Iran is taking advantage of the situation—something I wish to hell the United States would do.

“Iran, for reasons of its own, is supporting various Iraqi factions in the fight against ISIL, in direct and discreet ways that, frankly, the United States is not doing,” he adds. “What do you expect the Iraqis to do? Say no thank you?”

---

 

Regardless of the strength of Iraq’s Shia militias, or the assistance Iran provides, Islamic State will not be defeated in Iraq unless Sunni Arabs play a much larger role in the struggle.

The United States has taken some steps toward encouraging that to happen. It pressured Maliki to step down as prime minister, which he did last August, to be replaced by Haider al-Abadi, another Shia, but one who is less divisive and appears more committed to reaching out to Sunnis.

Abadi, however, is often described as well-intentioned but weak. “He is prime minister, but is not empowered to deliver on his political promises,” says Kawa Hassan, a visiting scholar at the Carnegie Endowment for International Peace, and a Middle East expert at Hivos, a Dutch NGO. Maliki, now vice-president, remains powerful, with allies in several government departments and ministries.

Related: Why targeted killings work against Islamic State

Meanwhile, the Popular Mobilization Forces have been accused of numerous sectarian abuses and atrocities. A report in February by Human Rights Watch details allegations of looting, arson, ethnic cleansing and murder. “It’s a pity to be a Sunni in Baghdad,” says Michael Weiss, co-author of  ISIS: Inside the Army of Terror. “They’ve been rounded up at checkpoints, thrown into dungeons, tortured, abused. It’s a sign of intimidation.”

But because the Shia militias are the most powerful ground force in Iraq, some analysts, such as Max Abrahms of Northeastern University, believe Iraq’s Western partners have little choice but to ally with them, and with their Iranian backers. He thinks America should make a similar choice in Syria and co-operate with Assad’s regime. “There’s no question Assad has more blood on his hands in Syria than does Islamic State, but, speaking as an American, I’m much more afraid of Islamic State than I am of Assad, because Islamic State wants to kill as many American civilians in the American homeland as possible, whereas the Assad regime does not.”

Weiss counters that allying with Assad in Syria and with Iran-backed militias in Iraq “is the easiest way to alienate the core constituency we need to turn against ISIS, namely, the Sunnis. Leaving aside the moral calculation, strategically—it’s just not going to work.”

Sunnis in both countries already have good reason to be wary of co-operation with the West. Assad and his proxies are responsible for most of the suffering in Syria. His army killed civilians with sarin gas. He suffered no repercussions. His air force continues to devastate civilian neighbourhoods. But because the world did nothing when he used chemical weapons, Assad knows it will do nothing when he bombs schools with conventional ones. “The United States has to convince Sunnis—and I don’t mean through rhetoric and posturing; I mean through tangible action—that their suffering, their plight, their dispossession and mass murder and ethnic cleansing matter,” says Weiss. “One way to do this would be to stop Assad’s air war against them.” Absent such action, expecting Syrian Sunnis to enlist as America’s ground troops against Islamic State is unreasonable, he says. “You can build up proxies. You can work with local actors on the ground. But you have to give them an incentive, and the incentive cannot just be, ‘Go and kill the terrorists on our behalf.’ Because, to them, the terrorist-in-chief is Assad and his regime.”

That America has not targeted Assad has led to speculation that it has at least a tacit understanding with Iran, with whom it is negotiating a nuclear deal: Do not harm Assad, because doing so would jeopardize those negotiations. “These narratives are beginning to trickle onto the population, and it’s beginning to affect their participation,” says Renad Mansour.

In Iraq, Sunnis who rose up against al-Qaeda during the surge of U.S. troops into the country were punished, rather than thanked by their government. “They don’t trust Baghdad and, unfortunately, they don’t trust us, either,” says Pregent, the former intelligence officer. “We promised them we would not abandon them, and we did. We can’t ask them to take up arms against ISIS when their government doesn’t see a difference between them and ISIS. We cannot ask you to take up arms and then say: ‘We’re going to be here for 18 months, guys. It’s all you afterward.’ They’ve seen this before.”

---

The recently announced deployment of 450 American trainers to Anbar is a small step in the right direction, because it will specifically target Sunnis. Such fighters might be integrated into Iraqi security forces and the Popular Mobilization Forces, possibly de-sectarianizing both. This, too, would require American leverage. “We’ve got to go in there and hug Baghdad closer than Tehran is,” says Pregent.

But sending more trainers, even ones charged with helping Sunnis, is unlikely to be sufficient in the short term. And with Islamic State now about an hour’s drive from Baghdad, Iraq doesn’t have a lot of time. “Nobody here is talking about Operation Iraqi Freedom 2,” says Knights, meaning a sequel to the large-scale 2003 invasion. Instead, he suggests deploying some 5,000 special forces who would embed with Iraqi units, including on the front lines, where they could directly call in air strikes.

Currently, America carries out about 15 air strikes a day, according to the New York Times, compared with about 800 a day during the 2003 Iraq war. There are fewer planes deployed now, but often they return to base without releasing their bombs, because pilots cannot identify targets or are not given approval to hit them. “Right now, our air power, we have to suck it through a straw,” says Knights. Having American spotters on the ground, he says, would mean sucking air power through a fire hose.

Related: How America-loving Kosovo has become a hub for Islamic State

Americans, in fact, are on the front lines, at least occasionally. A Kurdish commander at an outpost about 500 m from an Islamic State position in northern Iraq told Maclean’s in October that American advisers had been there earlier. But it appears American spotters are restricting their activity to the Kurdish north of Iraq, not in the south, where battle lines are frequently more fluid and dangerous.

Canadian special forces are also operating in northern Iraq, close enough to the front lines to have suffered a “friendly fire” fatality, and to have directly engaged in firefights with Islamic State. “The Canadians are doing more than we’re doing. The Australians are doing more than we’re doing. The strategy that the United States has put in place is so constrained. It’s so weak,” says Pregent.

The United States and its allies are doing less in Syria than in Iraq. Coalition air strikes notably helped to break Islamic State’s siege of the mostly Kurdish city of Kobani in northern Syria, an important symbolic victory. But America has not deployed advisers or trainers to work in Syria with Kurdish and other rebel forces confronting Islamic State there.

“Here’s the infuriating thing about this. If we Americans put one armoured brigade into Syrian Kurdistan, we’d be in Raqqa in a week and a half,” says Knights, referring to Islamic State’s de facto capital in northern Syria. Kurdish forces and rebel Free Syria Army fighters have captured territory nearby.

Afzal Ashraf, a consultant fellow at the Royal United Services Institute and former Royal Air Force officer who served at the Multinational Force headquarters in Iraq, argues for the use of Western or international airmobile forces—fighting troops that can deploy quickly to secure objectives, then hand them to Iraqis. They wouldn’t hold territory, and their lighter footprint may feel like less of an infringement on Iraq’s sovereignty than would the garrisoning of thousands of troops there, he says.

Even without deploying armour or airmobile units, there is more the United States and its allies could do in Iraq and Syria without approaching the level of intervention made previously in Afghanistan, and in Iraq a decade ago. This includes a greater use of on-the-ground spotters to call in air strikes, embedded special forces, and more night raids against high-value targets. A no-fly zone in Syria would give that country’s non-jihadist rebels protection from Assad’s barrel bombs and chlorine-gas attacks, and perhaps build American credibility among the country’s Sunni majority that will one day have a large say in how their country is governed.

But all these options require Obama to decide this is a war that must be won, that defeating Islamic State is worth the investment and sacrifices the task requires. It doesn’t appear he has made such a calculation. His priority is striking a nuclear deal with Iran. Defeating Islamic State, he says, will take a long time, and already his presidency is winding down. He inherited a war in Iraq he didn’t want. So will his successor.