Research

Research

Published

October 2023

Overview

This overview is intended to accompany the publication of our incidents of civilian harm related to the use of explosive weapons in the Gaza Strip since October 7, 2023.

Airwars applies the same general methodology to all conflicts monitored, which is available on our website here. This methodology has been assessed as highly conformant with Every Casualty’s Standards for Casualty Recording.

This methodology note explains in detail how we are applying our standard approach to monitoring civilian harm incidents in Gaza from explosive weapons use. Additional methodology notes will be released tailored to each other monitoring area, such as civilian harm in Israel from the actions of Hamas militants.

Read more about our casualty recording work

In June 2023, at the Human Rights Council’s 53rd session, the Office of the High Commissioner for Human Rights (OHCHR) presented its report on the impact of casualty recording on the promotion and protection of human rights, which included a number of references to Airwars’ casualty recording efforts.

Our work has also been researched by academics at the University of Sydney, Heather Ford and Michael Richardson, in their paper: Framing data witnessing: Airwars and the production of authority in conflict monitoring. You can read an executive summary of their findings here.

Dataset overview

All incidents are an aggregate of all open source accounts alleging civilian harm occurred in a particular moment in space and time. We consider our incidents as ‘live’, and should be updated over time to account for new information that may come to light, or may not have been identified by Airwars during the original research.

Defining an incident

Airwars uses an incident-based approach to document harm to civilians from explosive weapons use: each incident is defined as a moment in time and space where sources alleged that an explosive weapon led to the fatality or injury of civilians. This does not include incidents where only militants were killed or injured, however we do capture the details of militants killed or injured in events alongside civilians.

Airwars assumes civilian status unless otherwise specified. Any ambiguity on civilian status is captured within our casualty and belligerent ranges (see below).

Where the exact time of an incident is unknown, deaths and injuries may be aggregated under one event until more information comes to light.

Each incident is geolocated to the highest possible degree of accuracy by trained geolocation teams. Airwars additionally cross-checks existing geolocation efforts from the wider open source community, and includes credit to such work where applicable. Airwars also encourages feedback from open source experts on each assessment, and incorporates updates and feedback where possible.

Where locations cannot be identified, incidents will be aggregated until more information is known.

Source identification

As an all source aggregate, Airwars treats all sources as relevant to an incident depending on their proximity to the harm event and the level of detail available about the event. Airwars includes all sources regardless of political or ideological affiliation. Inclusion of sources in the archive should not be taken as an endorsement of the source.

All our sources for harm events in Gaza are identified by our trained team of Arabic-language researchers; we primarily identify sources in the language local to the area where the harm has occurred. Additional English-language sources are added depending on their relevance to the harm event. As we also evolve our focus to harm events reported by Hebrew-language resources, we will also mobilise our Hebrew language teams and apply the same local-language led approach.

All information is assessed, written up and archived within each assessment in order to allow users to conduct further investigation.

As we also work to identify attribution of harm, names of victims and victims demographics where known, we are reconciling names and other information published by official channels, such as the Palestinian Ministry of Health, with our incidents.

Information categorisation

Attribution

Strike status (while we use the term ‘strike’ here, this should also be taken to mean any action involving the use of an explosive weapon, including, for example, a VBIED). All incidents are cross-checked with official statements from the Israel Defense Forces, militant wings of Hamas and other Palestinian military groups, and are included as follows:

Declared: The declared strike classification applies to incidents in which a belligerent has accepted responsibility explicitly for carrying out a strike on a specific location or specific target. This includes any statement made by an official from the Israeli government or from official channels or websites linked to Palestinian militant groups (Palestinian Islamic Jihad, Hamas).

Likely: The likely strike classification applies to incidents in which at least two or more sources that reported on the incident explicitly attributed the strikes to a belligerent. In Gaza, Airwars’ Arabic-language researchers take typical local references and language used to describe Israel Defense Forces to identify local attribution, as the term ‘Israeli state’ is rarely explicitly referenced.

Contested: The contested strike classification includes incidents in which attribution of the strike to one belligerent was not agreed upon by all sources reporting on the incident, specifically when sources attribute a strike to both Palestinian forces (Palestinian Islamic Jihad, Hamas) and Israeli forces. Incidents where no sources attribute a strike and the resulting civilian harm to a specific belligerent are graded as contested until more information comes to light.

Single source claims: only one source was found with an explicit allegation of harm from a belligerent.

Civilian harm status

Confirmed: A specific belligerent has accepted responsibility for the killing or injuring of non-combatants or allied forces in a particular incident.

Fair: Where, in the view of Airwars, there is a reasonable level of public reporting of an alleged civilian casualty incident from two or more sources which includes attribution to a specific belligerent. In the context of reporting in the Gaza Strip, Airwars has found that explicit attribution to Israeli forces is rare, and instead is assumed by sources given the intensity of bombardment. Incidents are therefore assessed as ‘fair’ if at least one source mentions a belligerent in a credible harm incident. Incidents will be updated to reflect alternative attribution should that come to light at a later stage. In our ‘strike status’ category, as mentioned above, we will still refer to incidents with only one source explicitly attributing the harm to a belligerent as ‘single source claims’ to reflect the information environment.

Weak: These are presently claims seen by Airwars as ‘place-holder’ incidents until more information comes to light, given a lack of corroborating sources on either civilian harm or likely belligerent.

Contested: These occur where there are competing claims of responsibility within the sources: for example, sources may both attribute the harm to misfire from a Hamas rocket, or to an Israeli airstrike. There may also be inconclusive evidence supporting attribution to both belligerents, for example competing interpretations of munition fragments or blast impacts.

Discounted: This criteria is often applied to incidents that may have at first fallen into one of the above categories, but new information came to light since publication that suggests the original source material was incorrect. For example, more information may come to light about the identities of victims initially classified as civilians, that strongly suggests such individuals were combatants. Airwars researchers judge this information objectively and on a case by case basis.

Casualty ranges

As with all assessments, Airwars presents casualty figures recorded in an incident within a range.

All assessments include a minimum and maximum for both civilians and, if applicable, militants (‘belligerents’) injured and killed, taking the most recent figure from unique sources. For example, a source may initially say five civilians were killed. As more information on the incident becomes available, the same source may then say that the number rose to 10. In this case we would take the ‘10’ as this reflects the source’s updated understanding of events.

In the assessment summary, an explanation is offered as to the rationale behind the casualty range.

In cases where civilian status is contested, Airwars applies the minimum casualty range ‘0’ to both the civilian casualty field and to the belligerent field.

Information related to missing individuals, or civilians buried under the rubble following an attack is recorded in Airwars’ summary. The number of missing individuals would only be added to our death toll ranges if sources specified that those civilians have been killed. When new information comes to light regarding the fate of these victims, we would update the assessment accordingly, as well as our ranges.

Images and media referenced

As we are uploading images at pace for this project, all images have been automatically blurred to warn for graphic content given the high volume of graphic material. Less graphic images will be unblurred in due course.

Please also note that we include all images related to the sources identified, which can include images of militants.

All images are used under fair use as archival material. If you would like us to take down any images, please contact us at the info email listed below, using the subject line ‘Image use’.

Identification of victims

Airwars is also matching names of civilians identified through open source investigation with official names and IDs released by the Palestinian Ministry of Health where possible. As in other conflicts monitored, Airwars also records incidents of civilian harm where not all victims were named by sources, or where sources did not provide the victim’s full family name.

Impact

Airwars is tracking the reported impact of the use of explosive weapons on services or infrastructure relating to education, health or food supply. See below for more details on what is being included in each category:

    Education – Reported damage or destruction to education infrastructure (school, university, etc.) and/or injured or killed education staff Health – Reported damage or destruction to healthcare infrastructure or vehicles (ambulance, hospital, clinic, etc.), and/or healthcare staff (doctors, nurses, rescuers, etc.), killed or injured Food – reported damage or destruction to food infrastructure (food markets, agricultural land, food factory, water infrastructure, etc.) or machinery (tractor, etc.), and/or injured or killed civilians working in the food or agricultural sector, and/or livestock killed or injured

Updating our assessments

If you have lost loved ones in an incident listed on the Airwars site and would like to get in touch – or would like to ask us to remove a photo or to add another – then please also contact us at the info email listed below.

Errors and corrections

We strive for accuracy and transparency of process in our reporting and presentation. Our casualty monitoring is continually evolving, representing our best current understanding of any alleged incidents.

If you have new information about a particular event, or details we haven’t included; if you find an error in our work; or if you have concerns about the way we are reporting our data, then do please reach us at the info email listed below with the subject line ‘Assessment update’.

Contact: info [at] airwars [dot] org

▲ Satellite imagery, screenshot from Google Earth, October 2023

Published

July 2023

This note is intended to accompany the data and findings following our publication of all alleged US actions in Yemen during Obama’s first term 2009-Jan 2013. These are all published as assessments available in our archive.

This is the first phase of work, with the second phase covering Obama’s second term, between 2013-17, still undergoing assessment and investigation.

Accompanying methodology notes will be made available on our website upon publication.

Dataset overview

An incident

Airwars uses an incident-based approach to document alleged US strikes in Yemen: each incident is defined as a moment in time and space where sources alleged US involvement in a drone or airstrike that led to the fatality or injury of civilians or militants.

We have also coded for damage to civilian infrastructure in cases where alleged US strikes were also reported to have caused casualties and injuries. Our definition of ‘infrastructure’ is evolving, but to date accounts for any mention of the following terms by sources: hospital, school, agriculture marketplaces, gas facility, power station, water station, religious place.

Where the exact time of an incident is unknown, deaths and injuries may be aggregated under one event until more information comes to light. Each incident is geolocated to the highest possible degree of accuracy by trained geolocation teams, though in general for this dataset the geolocation process in Yemen has been challenged by a lack of available incident related imagery.

Where locations cannot be identified, incidents will be aggregated until more information is known. All incidents are considered ‘live’ in our archive, and can be updated and changed to account for evolving information.

The Bureau of Investigative Journalism: preserving the original archive Airwars made extensive use and citation of data collected on US actions in Yemen by The Bureau of Investigative Journalism (TBIJ), who handed over their dataset and research to Airwars when their drone monitoring programme came to a close.

TBIJ’s research also includes incidents where no fatalities were recorded, and focused predominantly on English-language media. Though we have not otherwise investigated strikes without casualty allegations, these incidents have nevertheless been kept within the Airwars archive to preserve the original record, and include the following standard statement: “This incident was identified by the Bureau of Investigative Journalism and has been included in Airwars’ database even though no casualties are mentioned.”

Source identification

We assessed all known open source claims of US strikes in Yemen since 2009 that resulted in casualties. This includes covering all open source allegations of death or injury of ‘civilian non-combatants’ or ‘militants’.

Sources were identified by our trained team of Yemen researchers, using an incident-based method to develop a continuously evolving list of sources for monitoring and investigating allegations of US strikes. Out of more than 2,600 sources archived, 35% are Arabic language sources, and 65% are English language sources. These sources originate from Twitter (approx 40%), local and international media/NGos (approx 58%) as well as Facebook.

We included any and all information relevant to a single incident of US strikes, whether this includes mention of civilian or militant harm or not, and regardless of the affiliation of the source.

We have additionally conducted a data mapping exercise coordinating with other civil society and documentation groups in order to cross-check our database with existing and similar datasets.

All information is assessed, written up and archived within each assessment in order to allow the user of the dataset to conduct further investigation if needed.

‘Strike’ terminology

The term ‘strike’ is used throughout this document and in our analysis to mean a kinetic action; each assessment further classifies this action depending on the level of detail provided by sources relating to the incident – for example, a naval bombardment, airstrike or drone strike.

Strike status

No US strikes have been officially declared by either CENTCOM or by the CIA between 2002 and 2017.

Declared: Between 2009-Jan 2013, due to the nature of both CIA and US military involvement in Yemen, and the lack of official acknowledgment by the CIA for their involvement, in lieu of public reporting on CIA actions, Airwars graded events as ‘declared’ strikes due to either:

1. A US government official statement acknowledging responsibility – for example, Attorney General Eric Holder 2. Comments made by anonymous US government sources to major media outlets (ABC News, Fox News, Wall Street Journal, Los Angeles, Washington Post, Al Arabiya, Long War Journal) 3. Reference in leaked diplomatic cables regarding US involvement in specific strikes released by Wikileaks. The cables revealed that the Obama administration was instructing the Yemeni government to take credit for strikes carried out by the US military.

Likely: The likely strike classification applies to incidents in which all of the sources reported on the incident attributed the strikes to US forces. In cases where all of the sources attribute the strike to the US military, and Yemeni officials have acknowledged to the media that the strike was carried out by the US, the strike is qualified as “likely” as well.

Contested: The contested strike classification includes incidents in which attribution of the strike to the US military was not agreed upon by all sources reporting on the incident.

This category has also been used to capture incidents using a broad inclusion criteria that reflects on the likelihood of US involvement, even if it was not explicitly mentioned by the sources. This includes cases of precision strikes on a moving target during night time or drone strikes, which fit a wider likely pattern of US military engagement as well as the documented limitations of the Yemeni military to carry out these types of strikes. We have chosen this broad approach given the high level of secrecy around US actions, as well as findings from investigators and in Wikileaks around the Obama administration explicitly instructing the Yemeni government to take credit for US strikes.

Each case should be treated with caution and read closely in order to understand this designation, which is outlined clearly within each assessment.

Single source claims: only one source was found with an allegation of harm from US forces.

Published

June 2023

 

On May 3rd, 2023, the US military announced that it had targeted a ‘senior Al-Qaeda leader’ in a strike in Syria. That same day, the White Helmets shared images from the scene where they were the first responders to the strike. They reported that a civilian had died: Lutfi Hassan Masto, a 60-year old farmer killed alongside his sheep.

More than a month later, CNN revealed that the US military had decided to open an inquiry into the incident – known as an ‘AR15-6’ – after doubts grew about the identity of the victim. A US official admitted “we are no longer confident we killed a senior AQ official.”

AR15-6s are the US military’s most detailed review of civilian harm allegations. The same procedure was initiated after a strike killed ten civilians in Kabul, Afghanistan on August 29th, 2021, when the US incorrectly identified aid worker Zemari Ahmedi as an Islamic State militant.

Earlier this year The New York Times published 66 partially redacted pages of that AR15-6, declassified by the Pentagon after a successful Freedom of Information Act request.

This is the latest document released by the NYT, adding to more than a thousand civilian harm assessments released relating to the US-led Coalition campaign in Iraq and Syria during the war against ISIS. While most of these assessments were shorter-form investigations intended to be more adaptable to high tempo situations, more than a dozen were full AR15-6s – each one dozens of pages long.

Over the past year, Airwars researchers have been coding and reviewing this tranche of declassified civilian harm assessments. With new funding from the Social Sciences and Humanities Research Council of Canada, we are now working in partnership with researchers at the Universities of Auckland, Minnesota, Newcastle and Ottawa to produce a comprehensive set of resources analysing this material due to be published over the next year.

In advance of that analysis, we are releasing a selection of excerpts of civilian harm cases chosen by our researchers, which place the decisions made in the Kabul strike in the context of a pattern of decision-making, and which can also inform understandings of the May 3rd strike on Lutfi Masto.

“An unknown heavy object”

Concerns with the quality of the surveillance feed have been observed by Airwars researchers throughout our analysis of the declassified Pentagon documents as having contributed to civilian harm incidents.

In the case of the Kabul strike, officials note in the AR15-6 that the surveillance feed “obscured the [ID] of civilians” and that the “trees and courtyard overhang limited visibility angles”.

In other cases, reviews of higher quality imaging prompted only by civilian harm allegations have also revealed that weapons originally perceived to be held by ISIS militants were in fact never there to begin with.

On November 12, 2015, one civilian – a child – was unintentionally killed in Ramadi, Iraq, after initially being perceived as “an unknown heavy object” during targeting.

The report admitted that when the video footage was reviewed on a 62” high definition TV, it was clear that the “person dragging a heavy object, was actually moving with a person of possible smaller stature”.

In the incident below, at least seven civilians were killed in Raqqa, Syria, on December 7th 2016 after Coalition forces incorrectly assessed the individuals were carrying weapons and wearing tactical vests. The civilian harm assessment revealed that this was despite the fact that the Coalition observed the building for six hours before it was struck.

“Driving at a high rate of speed”

Understanding how targets are selected is a common challenge for third parties reviewing the consequences of military actions. In both the AR15-6 and throughout many of the documents covering the US-led campaign in Iraq and Syria against ISIS, behavioural patterns were referred to as justifications for target selection: in many  harm incidents, the analysis of these patterns has proven deadly for civilians.

On May 11th 2017, declassified documents reveal that the speed at which a vehicle was driving was the reason for the strike. CENTCOM admitted to killing two civilians in the incident in August 2017. The declassified document, released more than three years later, shows that those killed were quickly identified as children by officials reviewing the post-strike observation footage.

The Kabul strike against Ahmadi’s vehicle similarly used the observation of driving habits and techniques to ascertain militant status, even though the reasoning for certain actions during driving could also be explained by a wide range of unobserved factors. In Ahmadi’s case, these factors included the need to visit different areas of his city in order to carry out errands.

During an interview conducted as part of the AR15-6, an official stated that the way that the driver also “carefully” and “gingerly” loaded up the car were all factors in the decision to strike. The New York Times later revealed that those actions were typical of  Ahmadi’s usual day at work, where he collected water to assist in humanitarian aid distribution.

“Unexpected collateral damage”

In Kabul, the AR15-6 notes a service member saying that “the explosions were massive” after the strike. Originally thought to be corroborating evidence for the supposed munitions held in the vehicle, military officials later noted that it is more likely that the secondary explosion was caused by a propane or gas tank.

From a strike that killed at least 70 civilians in Hawija that prompted a major investigation and policy reforms in the Netherlands, to a series of strikes on fuel trucks in Iraq that barely made headlines – secondary explosions appear throughout civilian harm assessments as a likely cause of death and injury.

In 2016, the US-led Coalition fired aerial rockets as warning shots over the civilian drivers of fuel trucks in Syria. However, the civilian casualty assessments reveal that these warning shots were rarely effective: some of the drivers simply swerved sharply to avoid the rocket fire or, in other cases, they left their vehicles, waited and then returned after a short time, presumably thinking that the immediate danger had passed.

The significant secondary explosions resulting from strikes on fuel trucks in many of these cases led to the deaths of the civilian drivers.

Other cases of secondary explosions occurred in more densely populated battlegrounds. On January 21st 2017, at least 15 civilians were killed when a strike caused “unexpected collateral damage” in a densely populated neighbourhood in Mosul, Iraq.

Four children were reported killed in the strike. An excerpt from an interview with one of the survivors, conducted by The LA Times, is included in the declassified assessment: “Why would they make a mistake like this? They have all the technology. This is not a small mistake”.

The assessment report is brief, reflective of the shorter form civilian harm assessments conducted throughout the war against ISIS. It does not recommend any further action, such as the opening of the more in depth AR15-6.

In total, Airwars has tracked at least 8,198 civilian deaths resulting from the actions of the US-led Coalition. The US-led Coalition has admitted to 1,437 casualties, with many of those incidents originating as Airwars referrals rather than proactive reviews by the United States military. To date, there remain 37 open cases of civilian harm allegations yet to be resolved.

The excerpts above and the forthcoming analysis reveal much about how the US military navigates the information environment: how it reads militant status within the behaviours of civilians, how secondary explosions are seen as unfortunate but unpredictable – even in the most densely populated areas – and how blurred surveillance footage can lead to children being mistaken for objects.

The US has begun a process of ostensibly reviewing these assumptions, with its new Civilian Harm Mitigation and Response Action Plan underway. Separately the Dutch Ministry of Defense has taken the unprecedented step of releasing a database of all weapon deployments by Dutch F-16s during their involvement in the Coalition campaign in Iraq and Syria. Other members of the Coalition have been less forthcoming. Later this year, Airwars is taking the UK Ministry of Defence to a tribunal to push for the release of their own civilian harm assessment in the single incident in which they admitted to having killed a civilian in eight years of intense campaign.

In the absence of full transparency and accountability for the civilians killed by the US-led Coalition, lessons cannot be appropriately learned for future operations. This failure to reckon with these past actions will continue to have devastating outcomes for civilians, as it has done for the victims of the Kabul strike in 2021, and likely too for Lutfi Masto and his family in Syria last month.

On June 29th, Airwars joined 20 other civilian protection and human rights organizations in calling on the US military to carry out an investigation into the incident in Syria that is robust, transparent, and accountable, with the hopes that this investigation will set a precedent for all future civilian harm allegations.

Authors: Anna Bailey-Morley, Stephen Pine, Alice Smith, and Anna Zahn

Volunteers who are supporting this project include: Anna Bailey-Morley, Stephen Pine, Alice Smith, Nasim Hassani, Arturo Gutierrez de Velasco, Reine Radwan, Nitish Vaidyanathan, and Austin Graff

 

 

Civilian harm in the Battle of Kharkiv

Heatmap of where civilians were harmed during the 79-day battle

As part of our on-going research mapping the human cost of war in urban environments, the following visualisation demonstrates all civilian casualty allegations in the densely populated city of Kharkiv between February 24th to May 13th 2022.

Airwars documented 68 incidents of civilian harm reported by local sources in Kharkiv city during the intense campaign, accounting for at least 83 casualties and up to 365 civilians injured.

In Kharkiv city, the highest number of incidents of civilian harm were reported in Saltivskyi, in Kyivsky and Shevchenkivskiy city districts, the northern and eastern parts of the city. In September 2022, Kharkiv Mayor Ihor Terekhov said in an interview that ‘there are some residential areas where there is nothing left’.

This visualisation accompanies Airwars’ casualty archive for Ukraine. Read more in our research brief here, and navigate the full archive here.

Heat map of all civilian harm incidents in the city of Kharkiv resulting from alleged Russian artillery and airstrikes during the Battle of Kharkiv, February 24th to May 13th 2022 at Street, Area, Nearby Landmark, and Exact Location accuracy.