Three weeks after journalists Azmat Khan and Anand Gopal published a damning New York Times account of civilian deaths caused by anti-ISIS airstrikes, the Coalition has yet to respond publicly. The investigation represented the first large scale, methodical ground survey of airstrikes and the harm they have caused in the war, in this case from three areas of Northern Iraq targeted by the Coalition. Civilian casualties were found to be 31 times more likely than the alliance was admitting.

Airwars recently sat down with Khan and Gopal in New York City to learn more about how they carried out their investigation. Below are highlights from the interview, lightly edited for clarity.

Airwars: No one had completed this kind of scientific study before during the conflict. Going into it, what did you expect to find? How did it compare to what you encountered?

Azmat Khan: We began planning this in February 2016. By April I was on the ground [In Iraq] and I was embedding with local forces, both Shia militias and then with Peshmerga forces, in certain frontline towns. I remember early on seeing how pivotal these airstrikes were in terms of re-taking cities.

There was one town that was really important to Shias, and so dozens of Shia militias had tried to retake it — Bashir — from where ISIS had launched mortars with chemical agents into a neighboring town, Taza. I watched several Shia militias based in Taza try and fail to retake Bashir, putting in all of their troops. Then the peshmerga agreed to try and retake it, and they put in maybe a fraction of the number of troops, but were supported by Coalition airstrikes in a way the militias weren’t, and Bashir fell within hours.

It really showed me the extent to which these airstrikes played a pivotal role in re-taking territory, but also the level of devastation. Many parts of Bashir were just up in smoke, when I visited the day after it was re-taken.

Unless you were on the ground, you couldn’t get a real sense of that scale. There’d been good accounts looking at civilian casualties — but nobody had looked at both those that successfully hit ISIS targets and those that didn’t, so a systematic sample. That’s what we teamed up to do. As more cities were being retaken, we though there’s an opportunity to do this.

I think what surprised me was I expected there to be vast discrepancies between the Iraqi Air Force’s civilian casualty rate and the Coalition’s, but the 1 in 5 statistic [1 in 5 airstrikes, they found, killed civilians], that appeared to be consistent across the board, in the entire sample of airstrikes, as well as those identified only as Coalition. That shocked me.

Anand Gopal: We had actually done a lot of reporting on airstrikes in civilian mass casualty incidents that didn’t make it into the piece — early on in Hawija; the takeover of Ramadi which was really devastating; Fallujah and Tikrit as well. I think initially we were both really shocked in the difference between what we were getting anecdotally and what was being reported. That’s sort of what inspired this initially. It took a little bit of time for us to figure out the best way to do this would be house to house – systematically.

Khan: It was hard to do that until October… that’s when they [Iraqi forces] were up to the Christian neighborhoods, Bartella was taken around this time. They had started the official campaign, but they weren’t in East Mosul.

Airwars: So It’s October 2016, that’s when you are starting the systematic sample?

Khan: That’s when I was first able to visit a significant number of airstrikes in downtown Qayyarah, a large enough sample to understand that this is possible, we can successfully do this. We came back in January, and then several more times.

On the ground

Airwars: How did you go about this work?

Gopal: For example in downtown Qayyarah you could see that every fourth house was destroyed. So we decided to start at one point in a town and go systematically and just go street by street. We went with various people, police officers and others.

Khan: First I went in with a local blacksmith; later on we went with federal police officers. We went in with many different people at many different times, just to make sure that we were protecting against any potential bias. .

Gopal: We also had to make sure we didn’t miss any of the destroyed places, so we got satellite imagery and [got an analysis of] the before and after satellite imagery to actually mark the destruction, for instance. Many of them are airstrikes, but some of them are demolitions. After ISIS was ejected, people come and demolish [an ISIS] house in retribution. Some of them were not the result of airstrikes at all.

Khan: Those are not in our sample. We excluded anything that was damaged from something else, like a demolition.

Gopal: Two challenges — one is to isolate those that were due to airstrikes from the rest, and the second is to figure out if it is Coalition or Iraqi.  

Airwars: So once you had these cases on the ground, did you match them with reported strikes?

Khan: I had early on gone in and done a calculation – I think there were 450 or so airstrikes officially labeled as “near Qayyarah”, the entire district, not even just downtown,  according to the Coalition’s daily summaries of airstrikes. Then we went through the civilian death casualty reports acknowledged by the Coalition, and found two civilian death reports, one of which was later amended to an injury.

And then we checked Airwars as well, to see whether any allegations matched, and I know there were at least two certain matches from our sample in downtown Qayyarah. Then we looked at open investigations to see if any might match. But of the 75 civilian deaths in that sample of 103 airstrikes, none of those 75 civilian deaths we found had been admitted to or acknowledged to by the Coalition, to date. And none of the 21 deaths from strikes that fell even just within 50 meters of a logged Coalition strike had been acknowledged by the Coalition.

Airwars: That’s mindboggling.

Airwars: And your sample, if anything, likely would have shown fewer civilian deaths — less than West Mosul?

Khan: Yes, the strikes in our 103 sample — which is how we arrived at the 1 in 5 rate — did not include West Mosul, and they occurred before the rule change in December [when the Obama administration made calling in airstrikes easier in support of Iraqi forces].

Gopal: For complex tribal and patronage reasons, strikes in the areas we looked at may be more accurate than those in, say Anbar province. This is because they are populated by the Jibburis, a large tribe whose members maintained a close relationship with US forces over the years. This dates back to a split between Jibburi sheikhs and Saddam Hussein in the late 1980s; by the 2003 invasion, these sheikhs had become one of America’s few Sunni allies, and they were rewarded with police and government posts.

This put them on the opposite side of al-Qaeda in Iraq, and by 2014 they had become known for their fierce resistance to ISIS. This means that the Coalition enjoyed a far better and more extensive informant network in northern Iraq than it did in Anbar. Given our focus on Ninevah and not Anbar, it is likely that if there was any geographic bias, it led us to undercount the civilian casualty rate.

‘Incredible devastation’

Airwars: What was it like on the ground when you talked to people?

Azmat: It was really tough because there is so much sensitivity involved; many are very traumatized.You also have to be very, very clear that because you are a journalist, you are not an aid worker, which is how many people can sometimes view Westerners. Even asking questions about losses — you have to be so careful about that, and it involved usually meeting with as many survivors or people who were eyewitnesses.

If anything, these people we interviewed skewed pro-government, because they were the ones who were allowed to return. All of our interviews happened with people who were living in these areas. We aren’t at a camp saying, “Tell me about your home?” We were at these places where the [strikes] had happened and we knew and could verify that these people live right next door.

Gopal: I know many children of ISIS members had probably been killed, but they are not in our sample because those families have fled or have been arrested and are kept in camps.

Airwars: What did the places you went to look like?

Gopal: Qayyarah was heavily damaged, I’d say. Every street – probably every three or four houses.

The Coalition’s own video of its attack on the Rezzo family home – since removed from its official YouTube channel

In all, Khan and Gopal found that among 103 airstrike cases they identified after house to house surveying, one in five had caused civilian deaths – a figure greater than 31 times what the Coalition itself had acknowledged in the survey areas.  

Khan: I’ve been to every one of the 103 [sites] and there were some distinctions. In Shura, by the time of liberation most of the airstrikes happened during the liberation period, not all but most. During the liberation period Shura was pretty depopulated; civilians had mostly left. So it was destroyed. It had just been shot up. Apparently ISIS fighters were staying in tunnels underneath homes. These houses, you could just find incredible devastation, but probably the least amount of civilian death because civilians had left at the time of the bombing.

In terms of verifying allegations, our work went far beyond interviews and analyzing satellite imagery. In addition to interviewing hundreds of witnesses, we dug through rubble for bomb fragments, or materials that might suggest ISIS use, like artillery vests, ISIS literature, sometimes their bones, because nobody would bury them.

We also got our hands on more than 100 sets of coordinates for suspected ISIS sites passed on by local informants. Sometimes we were able to get photos and videos as well. And ultimately, we verified each civilian casualty allegation with health officials, security forces, or local administrators.

The killing of a family

During the course of their research, Khan and Gopal learned of the case of Basim Razzo, who lost his wife and daughter, and his brother and nephew next door, when their homes were misidentified and bombed by the Coalition on September 20th, 2015. Basim barely survived the strikes, but set off on a long quest to have the US government admit its error.

The Coalition’s pre-targeting of Basim’s home – surveyed extensively, filmed by drones —  was what Khan and Gopal call “the best case scenario.” And yet even in this case – most strikes are given nowhere near the attention – the Coalition failed utterly to identify the structures as civilian in nature, and as having no connection whatsoever to ISIS.

In fact, the Coalition was so assured of the strike’s success that it uploaded a video of the attack online. Initially identified in the video as a car bomb factory, Khan and Gopal later learned the Coalition had internally identified it as an ISIS headquarters. It was none of these things.

Airwars: I want to talk about Basim. Why did you feel you had to tell this story through his own?

Khan: Basim’s case actually represented so many of our findings. It was important to us that we also use a character and a story that we could follow very closely through the process, and obviously a large part of that was that Basim was exceptional at documenting his own case very early on.

One of the biggest reasons is that he is the “best case” scenario. This is a man who has Western contacts, who speaks fluent English. There had been a [Coalition] video uploaded, so if anything should result in some kind of accountability, this is the best case scenario. This is a deliberate airstrike, not a dynamic one. It was an “ISIS headquarters,” which we were told, when I was at the CAOC (Combined Air Operations Center), a very senior intelligence officer told me that a target with one of the highest thresholds to meet is usually an ISIS headquarters… In so many ways Basim’s case was the ultimate, highest most deliberative process.

Airwars: When you say the best case scenario, you mean the best case on the Coalition side in terms of what intelligence they could have, and they still screwed up in such a fundamental way?

Gopal: if there was ever a strike they could get right, this would be the one. They have weeks to plan it, they have it as an ISIS headquarters. And so you know, if it’s an ISIS headquarters, the threshold for actionable intelligence has to be much higher. It can’t just be drone footage that doesn’t see women and children.

Airwars: They identified it as a headquarters and what was the genesis of that? In the story you talk about – it’s infuriating to read – that they didn’t see women and children.

Khan: One of the things I asked at the CAOC in Qatar was how do you identify local patterns of behavior. For example, I said, under ISIS a lot of women are not leaving their homes. So when you are looking at these pattern of life videos, are you taking these variable local dynamics into account? How do you distinguish for example when you are bombing in Iraq and one of these areas, how do you distinguish between patterns of behavior that are specific to Iraq vs. bombing in Afghanistan. What are the differences?

I was told that they could not get into a great deal of detail about ISIS’ “TTPs” — tactics, techniques, and procedures — their understanding of how ISIS generally operates.  They told me that these are developed through the intelligence community, in coordination with a cultural expert, but that they could not offer more detail about it.

Gopal: At the end of the day, it appears there are no consequences for getting it wrong, so there are no incentives to try to get it right.

Another piece of this is there were a number of strikes and incidents that appear to have violated principles of proportionality. Where you bomb an entire house and kill a bunch of civilians for one or two snipers. None of that ended up in the story, because we were, again, trying to interrogate the best case scenario.

Airwars: There’s a fighter on the roof, and they blow up the entire building. You’ve documented that as well?

Gopal: We have plenty of cases like that, but they were after the rule change in December 2016, (and not in the sample of 103 strikes), so a number of cases in late December early January in east Mosul where this was happening. We have a little sidebar in the story that mentions one instance very briefly —  for example, three civilians in one house were killed after at least one ISIS sniper broke into their house and used their roof.

Changing the rules

Note: After civilian deaths in Iraq and Syria began noticeably increasing during 2017, journalists began asking whether the rules of engagement in anti-ISIS operations had changed. Top US officials at first claimed they hadn’t, but later noted that a December rule change had made it easier for lower ranks to call in airstrikes at Mosul.

Airwars: You mentioned these rule changes. There was a lot of talk about what Rules of Engagement mean, and whether that changed in Iraq or Syria, or whether it’s a semantic conversation. What did you understand as having changed, and what did you see on the ground as a result?

Gopal: We didn’t use the term rule of engagement for this reason because it is a contentious term. Personally, and I’m speaking in a personal capacity, I do think that [the December authorization change] qualified as a rules of engagement change…

What we know is in December the number of people who had the authority to call in airstrikes was broadened. Commanders closer to the ground were able to call in airstrikes and both of us know from tracking this very closely on the ground that there was a marked difference.

We have to separate other differences, because there is a phase of battle change, they went from East Mosul to West Mosul which means you have a skyrocketing of civilian casualties. That’s going to happen because you are going there. There are questions of tempo and the number of strikes you are conducting. But from December 20th, from then immediately began to see a change. The number of cases we documented in East Mosul, just within 15 days it was like night and day so it was a real change on the ground.

Airwars: There were other variables, as you mentioned. From afar it’s not easy to splice out what is responsible for what.

Gopal: Right. The Battle for West Mosul didn’t start until the end of January, early February. But we saw this change in the casualties in December.

It is very clear after December 20th – the best actual experiment you can have is just look at the strikes in East Mosul. The neighborhoods before and after December 20th on either side of it are both in East Mosul. You look at the rate before and after and it’s countable. One can look at that and make an estimation. You can look at the Airwars archives before and after these dates, but just in east Mosul.

Airwars: The Coalition repeated over and over how precise the campaign was. What was your sense of this? Did you feel they were deluded? Did you feel they were obscuring the truth, did you feel that they just didn’t get it? Did you feel they were just trying their best?

Khan: Clearly, we have people who care a lot about this issue; they are not unfeeling. And one of the first things that they will often point out is, “We are not doing what the Syrian and Russian air force is doing.”

Sam: Do you feel it’s almost as if because this Russian campaign is happening at the same time they don’t have to be as careful because anything is better than what the Russians are doing?

Gopal: Of course Russian strikes deflect attention from what they are doing. The big difference is of course whatever the Russian air force is doing – which is horrible, undoubtable – they haven’t come out with a particular claim that they’ve killed some 400 civilians in 14,000 airstrikes— but the fact that the Coalition is making this claim means that it—it forces all of us journalists and researchers and academics to hold them to account to that.

More broadly I would say I think it’s in a way unfair almost to compare the two cases—the Syrian/Russian case and the Coalition case—because they are really the result of totally different histories and norms. What I mean is it used to be the case that – it was once accepted for the US to say target civilians. This is World War II in Dresden and firebombing Tokyo, the Korean War. Trump said the US wants to completely destroy North Korea; it would have been the second time they’ve done that. They would target civilians, they would target civilian infrastructure.

That shifted in Vietnam. Even though the laws of war had changed much before after Geneva, it shifted in Vietnam because of a really powerful antiwar movement that forced certain types of norms to be instituted within the military itself. That is the same paradigm we are living in now. The Coalition shouldn’t pat itself on the back that it’s not killing as many civilians as Russia. It’s the result of a process in which millions of people basically demanded and fought for that, against the wishes of the US military for generations.

‘Not a word from the Coalition’

Airwars: Have you had any official response since you’ve published this piece?

Khan: We had been in contact for about a year with questions, which they had been providing responses to. We had been checking coordinates from our sample in their logs. And more than a month before publication, we provided detailed information about all of the civilian casualty allegations that fell within 360 meters of logged Coalition coordinates: the names of dead of injured, photo evidence, contact information of survivors or witnesses or others they could reach on the ground,, before and after satellite imagery, and other evidence, and asked for any response or comment on any of them.

Although they answered other questions, we did not get a response about any of those allegations, and followed up a few times, including asking whether new investigations would be opened as a result of those allegations. And since the piece has been published, we still have not received a comment on that.

Gopal: We didn’t get a denial, we got nothing.

Khan: About the civilian casualty incidents not a word.

Anand: Not a word.