Research

Research

Published

October 2023

Overview

This overview is intended to accompany the publication of our incidents of civilian harm related to the use of explosive weapons in the Gaza Strip since October 7, 2023.

Airwars applies the same general methodology to all conflicts monitored, which is available on our website here. This methodology has been assessed as highly conformant with Every Casualty’s Standards for Casualty Recording.

This methodology note explains in detail how we are applying our standard approach to monitoring civilian harm incidents in Gaza from explosive weapons use. Additional methodology notes will be released tailored to each other monitoring area, such as civilian harm in Israel from the actions of Hamas militants.

Read more about our casualty recording work

In June 2023, at the Human Rights Council’s 53rd session, the Office of the High Commissioner for Human Rights (OHCHR) presented its report on the impact of casualty recording on the promotion and protection of human rights, which included a number of references to Airwars’ casualty recording efforts.

Our work has also been researched by academics at the University of Sydney, Heather Ford and Michael Richardson, in their paper: Framing data witnessing: Airwars and the production of authority in conflict monitoring. You can read an executive summary of their findings here.

Dataset overview

All incidents are an aggregate of all open source accounts alleging civilian harm occurred in a particular moment in space and time. We consider our incidents as ‘live’, and should be updated over time to account for new information that may come to light, or may not have been identified by Airwars during the original research.

Defining an incident

Airwars uses an incident-based approach to document harm to civilians from explosive weapons use: each incident is defined as a moment in time and space where sources alleged that an explosive weapon led to the fatality or injury of civilians. This does not include incidents where only militants were killed or injured, however we do capture the details of militants killed or injured in events alongside civilians.

Airwars assumes civilian status unless otherwise specified. Any ambiguity on civilian status is captured within our casualty and belligerent ranges (see below).

Where the exact time of an incident is unknown, deaths and injuries may be aggregated under one event until more information comes to light.

Each incident is geolocated to the highest possible degree of accuracy by trained geolocation teams. Airwars additionally cross-checks existing geolocation efforts from the wider open source community, and includes credit to such work where applicable. Airwars also encourages feedback from open source experts on each assessment, and incorporates updates and feedback where possible.

Where locations cannot be identified, incidents will be aggregated until more information is known.

Source identification

As an all source aggregate, Airwars treats all sources as relevant to an incident depending on their proximity to the harm event and the level of detail available about the event. Airwars includes all sources regardless of political or ideological affiliation. Inclusion of sources in the archive should not be taken as an endorsement of the source.

All our sources for harm events in Gaza are identified by our trained team of Arabic-language researchers; we primarily identify sources in the language local to the area where the harm has occurred. Additional English-language sources are added depending on their relevance to the harm event. As we also evolve our focus to harm events reported by Hebrew-language resources, we will also mobilise our Hebrew language teams and apply the same local-language led approach.

All information is assessed, written up and archived within each assessment in order to allow users to conduct further investigation.

As we also work to identify names of victims where known, we are reconciling names published by official channels, such as the Palestinian Ministry of Health, with our incidents.

Information categorisation

Attribution

Strike status (while we use the term ‘strike’ here, this should also be taken to mean any action involving the use of an explosive weapon, including, for example, a VBIED). All incidents are cross-checked with official statements from the Israel Defense Forces, militant wings of Hamas and other Palestinian military groups, and are included as follows:

Declared: The declared strike classification applies to incidents in which a belligerent has accepted responsibility explicitly for carrying out a strike on a specific location or specific target. This includes any statement made by an official from the Israeli government or from official channels or websites linked to Palestinian militant groups (Palestinian Islamic Jihad, Hamas).

Likely: The likely strike classification applies to incidents in which at least two or more sources that reported on the incident explicitly attributed the strikes to a belligerent. In Gaza, Airwars’ Arabic-language researchers take typical local references and language used to describe Israel Defense Forces to identify local attribution, as the term ‘Israeli state’ is rarely explicitly referenced.

Contested: The contested strike classification includes incidents in which attribution of the strike to one belligerent was not agreed upon by all sources reporting on the incident, specifically when sources attribute a strike to both Palestinian forces (Palestinian Islamic Jihad, Hamas) and Israeli forces. Incidents where no sources attribute a strike and the resulting civilian harm to a specific belligerent are graded as contested until more information comes to light.

Single source claims: only one source was found with an explicit allegation of harm from a belligerent.

Civilian harm status

Confirmed: A specific belligerent has accepted responsibility for the killing or injuring of non-combatants or allied forces in a particular incident.

Fair: Where, in the view of Airwars, there is a reasonable level of public reporting of an alleged civilian casualty incident from two or more sources which includes attribution to a specific belligerent. In the context of reporting in the Gaza Strip, Airwars has found that explicit attribution to Israeli forces is rare, and instead is assumed by sources given the intensity of bombardment. Incidents are therefore assessed as ‘fair’ if at least one source mentions a belligerent in a credible harm incident. Incidents will be updated to reflect alternative attribution should that come to light at a later stage. In our ‘strike status’ category, as mentioned above, we will still refer to incidents with only one source explicitly attributing the harm to a belligerent as ‘single source claims’ to reflect the information environment.

Weak: These are presently claims seen by Airwars as ‘place-holder’ incidents until more information comes to light, given a lack of corroborating sources on either civilian harm or likely belligerent.

Contested: These occur where there are competing claims of responsibility within the sources: for example, sources may both attribute the harm to misfire from a Hamas rocket, or to an Israeli airstrike. There may also be inconclusive evidence supporting attribution to both belligerents, for example competing interpretations of munition fragments or blast impacts.

Discounted: This criteria is often applied to incidents that may have at first fallen into one of the above categories, but new information came to light since publication that suggests the original source material was incorrect. For example, more information may come to light about the identities of victims initially classified as civilians, that strongly suggests such individuals were combatants. Airwars researchers judge this information objectively and on a case by case basis.

Casualty ranges

As with all assessments, Airwars presents casualty figures recorded in an incident within a range.

All assessments include a minimum and maximum for both civilians and, if applicable, militants (‘belligerents’) injured and killed, taking the most recent figure from unique sources. For example, a source may initially say five civilians were killed. As more information on the incident becomes available, the same source may then say that the number rose to 10. In this case we would take the ‘10’ as this reflects the source’s updated understanding of events.

In the assessment summary, an explanation is offered as to the rationale behind the casualty range.

In cases where civilian status is contested, Airwars applies the minimum casualty range ‘0’ to both the civilian casualty field and to the belligerent field.

Information related to missing individuals, or civilians buried under the rubble following an attack is recorded in Airwars’ summary. The number of missing individuals would only be added to our death toll ranges if sources specified that those civilians have been killed. When new information comes to light regarding the fate of these victims, we would update the assessment accordingly, as well as our ranges.

Images and media referenced

As we are uploading images at pace for this project, all images have been automatically blurred to warn for graphic content given the high volume of graphic material. Less graphic images will be unblurred in due course.

Please also note that we include all images related to the sources identified, which can include images of militants.

All images are used under fair use as archival material. If you would like us to take down any images, please contact us at the info email listed below, using the subject line ‘Image use’.

Identification of victims

Airwars is also matching names of civilians identified through open source investigation with official names and IDs released by the Palestinian Ministry of Health where possible. As in other conflicts monitored, Airwars also records incidents of civilian harm where not all victims were named by sources, or where sources did not provide the victim’s full family name.

Impact

Airwars is tracking the reported impact of the use of explosive weapons on services or infrastructure relating to education, health or food supply. See below for more details on what is being included in each category:

    Education – Reported damage or destruction to education infrastructure (school, university, etc.) and/or injured or killed education staff Health – Reported damage or destruction to healthcare infrastructure or vehicles (ambulance, hospital, clinic, etc.), and/or healthcare staff (doctors, nurses, rescuers, etc.), killed or injured Food – reported damage or destruction to food infrastructure (food markets, agricultural land, food factory, water infrastructure, etc.) or machinery (tractor, etc.), and/or injured or killed civilians working in the food or agricultural sector, and/or livestock killed or injured

Updating our assessments

If you have lost loved ones in an incident listed on the Airwars site and would like to get in touch – or would like to ask us to remove a photo or to add another – then please also contact us at the info email listed below.

Errors and corrections

We strive for accuracy and transparency of process in our reporting and presentation. Our casualty monitoring is continually evolving, representing our best current understanding of any alleged incidents.

If you have new information about a particular event, or details we haven’t included; if you find an error in our work; or if you have concerns about the way we are reporting our data, then do please reach us at the info email listed below with the subject line ‘Assessment update’.

Contact: info [at] airwars [dot] org

▲ Satellite imagery, screenshot from Google Earth, October 2023

Published

July 2023

This note is intended to accompany the data and findings following our publication of all alleged US actions in Yemen during Obama’s first term 2009-Jan 2013. These are all published as assessments available in our archive.

This is the first phase of work, with the second phase covering Obama’s second term, between 2013-17, still undergoing assessment and investigation.

Accompanying methodology notes will be made available on our website upon publication.

Dataset overview

An incident

Airwars uses an incident-based approach to document alleged US strikes in Yemen: each incident is defined as a moment in time and space where sources alleged US involvement in a drone or airstrike that led to the fatality or injury of civilians or militants.

We have also coded for damage to civilian infrastructure in cases where alleged US strikes were also reported to have caused casualties and injuries. Our definition of ‘infrastructure’ is evolving, but to date accounts for any mention of the following terms by sources: hospital, school, agriculture marketplaces, gas facility, power station, water station, religious place.

Where the exact time of an incident is unknown, deaths and injuries may be aggregated under one event until more information comes to light. Each incident is geolocated to the highest possible degree of accuracy by trained geolocation teams, though in general for this dataset the geolocation process in Yemen has been challenged by a lack of available incident related imagery.

Where locations cannot be identified, incidents will be aggregated until more information is known. All incidents are considered ‘live’ in our archive, and can be updated and changed to account for evolving information.

The Bureau of Investigative Journalism: preserving the original archive Airwars made extensive use and citation of data collected on US actions in Yemen by The Bureau of Investigative Journalism (TBIJ), who handed over their dataset and research to Airwars when their drone monitoring programme came to a close.

TBIJ’s research also includes incidents where no fatalities were recorded, and focused predominantly on English-language media. Though we have not otherwise investigated strikes without casualty allegations, these incidents have nevertheless been kept within the Airwars archive to preserve the original record, and include the following standard statement: “This incident was identified by the Bureau of Investigative Journalism and has been included in Airwars’ database even though no casualties are mentioned.”

Source identification

We assessed all known open source claims of US strikes in Yemen since 2009 that resulted in casualties. This includes covering all open source allegations of death or injury of ‘civilian non-combatants’ or ‘militants’.

Sources were identified by our trained team of Yemen researchers, using an incident-based method to develop a continuously evolving list of sources for monitoring and investigating allegations of US strikes. Out of more than 2,600 sources archived, 35% are Arabic language sources, and 65% are English language sources. These sources originate from Twitter (approx 40%), local and international media/NGos (approx 58%) as well as Facebook.

We included any and all information relevant to a single incident of US strikes, whether this includes mention of civilian or militant harm or not, and regardless of the affiliation of the source.

We have additionally conducted a data mapping exercise coordinating with other civil society and documentation groups in order to cross-check our database with existing and similar datasets.

All information is assessed, written up and archived within each assessment in order to allow the user of the dataset to conduct further investigation if needed.

‘Strike’ terminology

The term ‘strike’ is used throughout this document and in our analysis to mean a kinetic action; each assessment further classifies this action depending on the level of detail provided by sources relating to the incident – for example, a naval bombardment, airstrike or drone strike.

Strike status

No US strikes have been officially declared by either CENTCOM or by the CIA between 2002 and 2017.

Declared: Between 2009-Jan 2013, due to the nature of both CIA and US military involvement in Yemen, and the lack of official acknowledgment by the CIA for their involvement, in lieu of public reporting on CIA actions, Airwars graded events as ‘declared’ strikes due to either:

1. A US government official statement acknowledging responsibility – for example, Attorney General Eric Holder 2. Comments made by anonymous US government sources to major media outlets (ABC News, Fox News, Wall Street Journal, Los Angeles, Washington Post, Al Arabiya, Long War Journal) 3. Reference in leaked diplomatic cables regarding US involvement in specific strikes released by Wikileaks. The cables revealed that the Obama administration was instructing the Yemeni government to take credit for strikes carried out by the US military.

Likely: The likely strike classification applies to incidents in which all of the sources reported on the incident attributed the strikes to US forces. In cases where all of the sources attribute the strike to the US military, and Yemeni officials have acknowledged to the media that the strike was carried out by the US, the strike is qualified as “likely” as well.

Contested: The contested strike classification includes incidents in which attribution of the strike to the US military was not agreed upon by all sources reporting on the incident.

This category has also been used to capture incidents using a broad inclusion criteria that reflects on the likelihood of US involvement, even if it was not explicitly mentioned by the sources. This includes cases of precision strikes on a moving target during night time or drone strikes, which fit a wider likely pattern of US military engagement as well as the documented limitations of the Yemeni military to carry out these types of strikes. We have chosen this broad approach given the high level of secrecy around US actions, as well as findings from investigators and in Wikileaks around the Obama administration explicitly instructing the Yemeni government to take credit for US strikes.

Each case should be treated with caution and read closely in order to understand this designation, which is outlined clearly within each assessment.

Single source claims: only one source was found with an allegation of harm from US forces.

Published

May 2023

Written by

Airwars Staff

This overview is intended to accompany our Research Brief ‘Patterns of civilian harm from alleged Russian actions in Kharkiv oblast’, our conflict overview page, and our archive.

This approach was originally developed as we documented conflicts across Libya, Syria, Iraq, Yemen, Somalia and the Gaza Strip, and closely aligns with global standards on Casualty Recording.

What is an ‘incident’? 

Airwars uses an incident-based approach to document civilian harm: each incident is defined as a moment in time and space where sources reported death or injury to civilian non-combatant populations. See more on sourcing below.

Where the exact time or location of an incident is unknown, civilian deaths and injuries may be aggregated under one event until more information comes to light. Each incident is geolocated to the highest possible degree of accuracy by trained geolocation teams. Where locations cannot be precisely identified, civilian harm incidents will be aggregated until more information is known.

Our threshold of geolocation accuracy for including an incident in our archive is at sub-district level.

All incidents are considered ‘live’ in our archive, and can be updated and changed to account for evolving information.

Who are our sources?

For our work casualty recording, we assessed all known open source claims of civilian non-combatants killed or injured by all parties using explosive weapons in Kharkiv oblast.

Our sources were identified by our trained team of Ukrainian researchers, using an incident-based method to develop a continuously evolving list of sources for monitoring and investigating allegations of civilian deaths or injuries. These sources include Telegram, Twitter, Facebook, and local and international media or organisations, in Ukrainian, Russian and English language.

We routinely conduct data mapping exercises to ensure that we are coordinating with other civil society and documentation groups working in a similar field.

To date, similar aggregate databases covering casualty records in Kharkiv include: Bellingcat, Eyes on Russia-Center for Information Resilience, Attacks on Health Care in Ukraine Project, Hala systems, Armed Conflict Location and Event Data (ACLED), Victims Memorial, Ukraine War Archive, Kharkiv Human Rights Protection Group and Helsinki Human Rights Union/Tribunal For Putin (T4P) initiative.

These databases are referenced throughout our assessments, to facilitate information sharing and data reconciliation across the diverse range of documentation efforts.

Note that Airwars’ mention of an incident recorded by another open source dataset does not mean that Airwars findings precisely match what has been recorded by that organisation. In certain cases, findings may differ due to the sources used or due to differences in methodology.

Managing dis/misinformation

As in all conflicts we monitor, we include any and all information relevant to a single incident of civilian harm, regardless of the affiliation of the source. In other words, if a source includes reference to the civilians killed or injured in a single incident it will be included in our assessment. If a source includes only generic information without being linked to a single incident, it will not be included.

All information is assessed, written up and archived within each assessment in order to allow the user of the dataset to conduct further investigation if needed.

Civilian casualty categories

Airwars has developed a unique methodology to categorise civilian harm incidents according to the nature of information identified in relation to the incident. This approach allows users of the archive to quickly understand the information environment relating to each allegation.

    Fair: all sources agree civilian harm occurred as a result of the actions of one belligerent (i.e. all sources agreed that Russian forces killed civilians in a single strike) Contested: not all sources agree on who was responsible for the civilian harm (i.e. some sources alleged harm resulted from Russian actions, some sources alleged harm resulted from Ukrainian forces actions). Weak: the allegation of harm came from only a single source, with little unique information circulated amongst open sources. This category may change to ‘fair’ or ‘contested’ if more information comes to light. For example, incidents qualified as ‘weak’ in occupied areas, where access to information has been restricted by Russian forces, may later be reassessed after new details are revealed. Discounted: Incidents where the original assessment of civilian casualties is later proven inaccurate, due to new information that comes to light.

Additional categories and notes

Personal information on civilians harmed

Names and personal information related to civilians harmed whose identities are known have not been included in our public archive, due to on-going security fears and with respect to the Ukrainian Law on Protection of Personal Data, along with other regulations. Airwars maintains a secure private archive of individuals named, should the situation evolve.

Damage to infrastructure

We have coded for damage to civilian infrastructure in cases where civilian death or injury was also reported. Our definition of ‘infrastructure’ is evolving, but to date accounts for any mention of the following terms by sources: healthcare infrastructure, school, agriculture, humanitarian aid distribution, humanitarian evacuation, religious institution, marketplace, gas facility, power station, and water station.

Challenges and limitations

On casualty recording in Ukraine and in Kharkiv region

The Ukrainian government stopped publishing national casualty estimates four days into the war. The United Nations, currently the sole official source for those figures, does not provide details of casualties per region. In the beginning of May 2023, the UN’s estimate found confirmed evidence of only 8,800 Ukrainian civilians killed in 15 months in all of Ukraine, mostly resulting from the use of explosive weapons. The UN has repeatedly admitted this is an underestimate, while Ukrainian war prosecutors estimate the real toll may be 10 times higher.

Airwars’ Ukraine team has found that a number of local sources often referred to authorities regarding casualty reporting and casualty numbers, and were less likely than in other conflicts monitored by Airwars to provide separate grassroots estimates of civilian deaths. This might be explained by the fact that even prior to the full-scale invasion, Ukraine has had functioning official security, forensic and medico-legal structures trusted by civilians to internally record civilian harm. Fears of occupation and retaliation in Kharkiv oblast during the time of reference could also explain less willingness from sources to publish open source details about incidents.

Our team has noted limited accounts of civilian injuries reported in local media, with sources instead often referring to civilians ‘pulled from the rubble’. Without clear mention of injuries, we have not included those rescued from the rubble in our range of civilians injured. However, we have captured this information in our summary of the incident. This is a datapoint we are exploring as an information proxy for physical harm.

On the identification of victims and missing individuals 

In a number of incidents, our researchers encountered one major gap in the information publicly released – the identities of the victims. In Ukrainian language reporting there has been few mention of names of victims, and limited information related to their gender, age or occupation. There are a number of potential reasons for this gap but a key factor is the Ukrainian Law on Protection of Personal Data, along with other regulations, applied by authorities to protect civilians’ identities. This allows the release of personal information only when approval has been given by the victim or their family. Other reasons might involve security risks related to documentation of civilian harm within areas of active combat such as Kharkiv, as well as the difficult or forbidden access for civilians to destroyed or collapsed buildings in urban areas when looking for their loved ones.

The identification of civilians killed in the conflict has also been particularly challenging in Kharkiv oblast given that many areas in the region were occupied by Russian forces for several months. In October 2022, Ukrainian authorities reported that they found hundreds of bodies in newly retaken areas. When causes of death were not known, as many families are still waiting for the results of DNA tests and additional investigations to know the fate of their loved ones, these victims were not included in our casualty totals. Some experts have said that it could take years to find and identify civilian victims, with Kharkiv Human Rights Protection Group estimating that nearly 2,000 individuals remain missing in Kharkiv oblast alone.