Welcome to Volume 6 of Metaculus Mondays!
There have not been any question resolutions since the last volume was released, but we can probably kiss our closest EU vaccination prediction goodbye (75M doses by April 1). If you haven’t followed the story, many European Union member states decided to pause AstraZeneca vaccinations this week after some health officials realized 37 of 17,000,000 vaccinated people–or 0.0002% of those vaccinated–had a reported blood clot.
Was this a smart move? Probably not. As cited in a 2018 article in Thrombosis Research (Blood Clot Research), 543,454 people die from blood clots in the EU every year (or 45,288 people per month). 37 blood clots per 17 million applied to the entire EU population would mean 971 blood clots (not deaths) over the past 2.5 months. (Edit: Turns out the 17M vaccinated includes people in the UK, so this is more like 1093 blood clots.)
Mini prediction retrospective: We underestimated the EU’s ability to...well, do the most EU thing: Pause vaccinations during a pandemic without compelling, significant evidence.
Another one of our forecasts is set to resolve shortly, but we expect to get this one right: Whether U.S. troops will remain in Afghanistan after the May 1st deadline. We gave a 90% likelihood troops stay. Since that forecast, seemingly every outlet across the political spectrum has told Biden the reasons why troops should stay–at least for now, they say–and it came out we had 3,500 troops left in Afghanistan and not 2,500. If we had the chance to update our forecast before it closed today, we would’ve upped it to a 92.5% likelihood, but we forgot. Clay swears the countdown timer deceived him. ?
Moving ahead to this week’s volume, we want to provide some updates on the Metaculus community and forecast how the conflict in Yemen will unfold throughout the rest of 2021. But first, we want to move past our usual uncontroversial topics—US rejoining the JCPOA, future of Saudi-Israeli relations—and tackle a question that has implications for global health regulations, intergovernmental institutions, and relations between states.
What was the source of SARS-CoV-2?
Where Did Covid-19 Originate?
No one knows the origin of SARS-CoV-2. And we are not guaranteed to ever know for sure. Although we first identified Ebola in 1976, we still do not know its exact origins and it took over a decade to identify the origins of COVID-19’s younger brother, SARS. And for months China blocked (and continues to interfere with) investigations into the origins of COVID-19.
There may be no definitive thesis yet, but there are four major hypotheses worth mentioning:
- The virus evolved in nature and was transmitted to humans from animals
- The virus evolved in nature and was transmitted to humans when researchers attempted to collect virus samples from animals
- The virus was collected from either animals or humans, genetically modified during lab research, and was released by accident
- The virus was genetically engineered as a bio-weapon and was deliberately released
Although preliminary findings from the World Health Organization (WHO) indicate uncertainty of the virus’ origins, it is the WHO’s opinion that the “most likely” possibility was that SARS-CoV-2 originated in animals. This would indicate a zoonotic origin (hypothesis 1 and 2). And while uncertain of the source, the WHO is mostly certain what the source was not: A lab leak–or hypothesis 3.
The Metaculus community appears to agree, at least based on their forecasts to these two questions:
“Will it turn out that Covid-19 originated inside a research lab in Hubei?”
- Community Median: 13% Likelihood
“Credible claim by 2024 that COVID-19 likely originated in a lab?”
- Community Median: 19% Likelihood
According to the December 30, 2020 NPR story, “Even If It’s ‘Bonkers,’ Poll Finds Many Believe QAnon And Other Conspiracy Theories”, 40% surveyed American adults believed “coronavirus was made in a lab in China even though there is no evidence for this” (emphasis added).
So we’re not about to make a prediction indicating otherwise. Right?
...
...
Right?
Rootclaim-Based Analysis
How does one begin forecasting the origins of Covid-19? Unlike nearly all our previous forecasts, this one pertains to a past event. Instead of predicting the future, we are investigating the past and assigning likelihood probabilities to each potential hypothesis.
The hypotheses we are seriously examining are whether, SARS-CoV-2:
- Evolved in nature and was transmitted to humans from animals -> Pure Zoonotic
- Evolved in nature and was transmitted to humans when researchers attempted to collect virus samples from animals -> Zoonotic Collection
- Was collected from either animals or humans, genetically modified during lab research, and released by accident -> Lab Leak
To analyze these hypotheses, we are following the framework and general evidence established by Rootclaim. Rootclaim is a site “focused on exposing the truth on many issues in public discourse using probabilistic inference.” In other words, Rootclaim picks (mostly) controversial events with unclear hypotheses behind their cause and tries to determine the truth using Bayesian statistics, crowd forecasting, and clear reasoning and evidence. Rootclaim has a limited, yet generally positive track record despite making what would otherwise be thought of as controversial, and incorrect, claims such as:
- 87% chance opposition forces in Syria (Liwa al-Islam) carried out the August 21, 2013 Ghouta chemical attack.
But controversy and a feeling of incorrectness does not mean prohibition. Not only that, but Rootclaim provides a full accounting of their reasoning and calculation, meaning we can use their framework to reach our own conclusions.
By removing the biochemical weapon probability and merging hypotheses 1 and 2 (zoonotic and zoonotic collection), we start with the following base-rate probabilities as determined by Rootclaim’s analysis:
- 99.08% Origin is Zoonotic (77.19% Pure Zoonotic; 21.89% Zoonotic Collection)
- 00.92% Origin is Lab Leak
The next step is to update this base-rate likelihood given the effect collected evidence has on the different hypotheses. Given that we are following much of Rootclaim’s analysis, we will only comment when we diverge from their analysis.
The first piece of evidence offered by Rootclaim is the contagion and mortality of Covid-19 which is unimportant to our forecast since we are not seriously considering the likelihood of a bio-weapon origin. At the same time, we want to expand the first piece of evidence to further include the overall genetic structure of the virus to account for other analysis we find worth including. Although many of the factors identified by Trevor Bedford in late February of 2020 that made a lab-leak incident unlikely have since changed (for instance, research in May by the Chinese CDC found SARS-CoV-2 did not originate in a Wuhan seafood market), Bedford identified the outgrowth of the virus being more indicative of a random-draw event.
As a result, we multiply the Zoonotic outcomes 1.1x to account for this fact and normalize the probabilities, resulting in the new likelihoods:
- 99.16% Zoonotic (77.25%; 21.91%)
- 0.84% Lab Leak
The outbreak location and virus sources near Wuhan were the second and third pieces of evidence identified by Rootclaim. The former reduced the likelihood of Zoonotic origin, while the latter reduced the likelihood of both:
- 96.29% Zoonotic (21.71%; 74.58%)
- 3.71% Lab Leak
The next two pieces of evidence are the most critical to the lab leak theory and center around the chimeric nature of SARS-CoV-2 and a furin cleavage found in the virus.
To make a very complicated amount of reading somewhat simple, SARS-CoV-2 has a chimeric structure, with the virus being 96% similar to RaTG13, however in a particularly important part of the virus it is less than 80% similar while being 99% similar to a coronavirus found in pangolins (both of these viruses were at the Wuhan Institute of Virology). An indication of a chimeric structure does not mean Covid-19 was engineered since chimeric viruses do emerge naturally, however, given the circumstances it is less likely to have done so.
At the same time, we are less confident in Rootclaim’s effect analysis, and feel there is too much confidence. One piece of evidence for our lack of confidence (Rootclaim’s confidence is 90%) is that recently researchers in China found a small-cluster of viruses similar to SARS-CoV-2, with “one of the viruses isolated from a Rhinolophus pusillus bat shared 94.5% of its genome with the pandemic virus.” As a result, we lowered our confidence to 70%, dropping the Rootclaim’s multiplier from 1/10 to ⅓.
The other peculiar aspect of SARS-CoV-2 is its furin cleavage, one of the factors that makes the virus so transmissible and more deadly in animals (but also more complicated; as the same Nature article shows, the removing the furin cleavage makes the virus less transmissible in E6 laboratory cells.) Given some uncertainty injected by recent studies into the furin cleavage in SARS-CoV-2, we lower the confidence in this effect analysis from Rootclaim’s 90% to 80%, thereby lowering the reduction from 1/10 to ⅕.
The effect of these two pieces of evidences is we now have the following probabilities:
- 60.89% Zoonotic (13.73%; 49.42%)
- 39.11% Lab Leak
For the remaining evidence, we kept the evidence and effect analysis done by Rootclaim, leading to the final likelihoods of:
- 64.32% Zoonotic Origin (6.96% Pure Zoonotic; 57.36% Zoonotic Collection)
- 35.68% Lab Leak Origin
However, we wanted to further include three pieces of non-quantitative evidence into our forecast using the 51-49 rule (meaning if we find the evidence in favor of one hypothesis we give it 51% weight, while we give the alternate 49% weight before normalizing).
Initial Event Forecast
We then round both numbers down, and gave the accumulated 1% likelihood to all other outcomes including the possibility of a bio-weapon origin, creating the final likelihoods for each hypothesis of:
- Zoonotic Origin (Pure and Collection): 61%
- Lab Leak Origin: 38%
- Other Origin: 1%
Now that we’ve calculated the odds that SARS-CoV-2 originated in a lab—with the nearly certain candidate (>99%) being the Wuhan Institute of Virology—we now calculate the probabilities for the two Metaculus questions above.
Metaculus Forecast: Q1
In truth, there was no need for any of this analysis for the first question...because my god is it busted.
Seriously.
The first question was as follows: “Will it turn out that Covid-19 originated inside a research lab in Hubei?” Makes sense, right?
Well, before we comment on the resolution criteria, let’s pause for a second. Try to think how you would determine whether or not it turned out “Covid-19 originated inside a research lab in Hubei?” Write down your resolution criteria in the comment section if you have confidence yours will be better than the actual one.
Last chance.
Okay, the directly-quoted resolution criteria for this question is:
- If at any time after the date of May 1, 2020 the community prediction is > 97% or < 3%, the question closes. Then, with 90% probability (as called by a quantum RNG), resolves positively or negatively, respectively. (With 10% probability the question is referred to the below committee.)
- Otherwise, on or about the close/resolve date of May 1, 2021, the question will be decided by unanimous vote of a council of three people as to whether the proposition listed above is true, at 50+% credence (i.e. more true than its negation), resolving ambiguous in the case of disagreement. The council of three will be chosen by quantum mechanical random numbers from a list of 12 people that will be composed by the author around the time of 2020-05-01, and held secretly until the time of question resolution.
Probably should’ve posted that comment.
Honestly, what a horrible question, with horrible^2 resolution criteria.
First of all, we did not originally notice the resolution date of this question was May 1, 2021. Why is that absurd? Because the date expects we will know the origins of a pandemic a year and a half after its onset.
But the resolution date is but a scratch. The resolution criteria is where the W I L D stuff is at. How are we to determine if Covid-19 originated inside a research lab in Hubei? According to this question, in two ways you probably never expected:
- Either a whole lot of people come to the same, strong conclusion that Covid-19 did originate (>97% median) or did not originate (<3% median) in a lab [or voted in a way that matched voting with a strong conviction one way or another]; OR
- The Politburo ☭ decides. (Technically the alternative is that 3 of the 12 people the question-poster pre-selected have to unanimously agree or disagree on lab origins, but–ya know–same thing.)
While we strongly feel in this time horizon the true probability is around 1%, there is a secret committee deciding the question. Your fate is never certain in those instances.
Therefore, we predict a 25% chance that by May 1 it will turn out SARS-CoV-2 originated inside a research lab in Hubei.
I mean we only get slightly fewer points this way anyways.
What would make us reconsider?
- If the resolution criteria changed. Assuming the change was not even more absurd, we would lower our forecast to either 1% or 2%
- If the community began approaching 3% or 97%, then we ???
Metaculus Forecast: Q2
Thankfully, the second Metaculus question is much much much better: “Credible claim by 2024 that COVID-19 likely originated in a lab?”
Not only is the 2024 resolution date much more reasonable, 4 years after the onset, but the criteria also makes sense. This question resolves positively if, before the end of 2024, at least two of the following public health agencies claim “it is more likely than not” that Covid-19 originated in a Chinese virology laboratory:
- CDC
- European CDC
- WHO
- Chinese CDC
- Centre for Health Protection
- Robert Koch Institute
- The National Institute of Infectious Diseases
- Public Health England
- The National Centre for Infectious Diseases
- CDC Korea
- The Public Health Agency of Canada
More specifically, the question requires that at least two of these agencies “broadly state that it is more likely than not that COVID-19 originated from a Chinese virology or biology laboratory, after having been released accidentally or deliberately. Synonyms for probability assessments must be considered by an admin to be broadly consistent with at least a 50% chance. Examples of such synonyms include "probably", "likely", "with high probability" and "almost certainly".”
With this much clearer resolution criteria, we feel comfortable making a real forecast. Using our previously analyzed likelihood of 38%, we then cut this number in half for two reasons:
- Time (less significant)
- Politics (more significant)
Therefore, we forecast there is a 19% likelihood that by 2024 at least two of the above public health agencies will claim SARS-CoV-2 likely originated in a Chinese virology lab.
What would make us reconsider?
- Any change in our understanding of the facts underpinning the different steps in analysis, with the main factors being into the mosaic structure of the virus and the importance and structure of the furin cleavage.
- A change in global geopolitical relations with China, with a trend towards positive meaning the odds go down and a trend against meaning the odds go up (as political friction to positive resolution reduces)
3/22/21: 48.45% Lab Leak
| New / Revisited Evidence | Effect |
|---|---|
| WHO-led Team Believes Wildlife Farms | Weak: 51%-49% Weight Towards Zoonotic |
| Change in Confidence of Chimera to 80% | Moderate: Lab Leak |
| 80,000 Hours Interview with Andrew C. Weber | Weak: 51-49, Lab Leak |
Since we first made this forecast last week, we’ve done further research on the subject and added a few new pieces of evidence affecting our overall forecast.
The first piece of evidence comes from a WHO-led mission, which believes “wildlife farms in southern China are the most likely source of the COVID-19 pandemic” according to NPR. Although there is some evidence provided behind this hypothesis (mainly that China shut down the exotic farms in southern China that could have been a bed for SARS-CoV-2 to emerge), the messenger (who reportedly has a conflict of interest) and lack of concrete evidence makes this evidence have a weak overall effect in favor of a zoonotic origins.
Moreover, we’ve further done more research and read further analyses which increases our confidence in the chimeric analysis conducted by Rootclaim from 70% to 80%--decreasing the effect multiple on Zoonotic origins from ⅓ to ⅕.
Finally, a comment in the Metaculus question pointed us to an interview with Andrew C. Weber, a former Assistant Secretary of Defense for Nuclear, Chemical & Biological Defense Programs and Deputy Coordinator for Ebola Response. In a podcast interview, Weber--an Obama appointee--stated a 50-50 confidence that SARS-CoV-2 was a result of a lab leak incident. We found this to be weak evidence in favor, showing further institutional-type buy-in.
The overall result is increasing our overall confidence that SARS-CoV-2 was a result of a leak-leak incident to 48.45%.
Accounting for political and other factors, we still apply a 50% discount to this probability, therefore increasing our likelihood of a lab-leak origin for COVID-19 from 19% to 24%.
Clay Graubard
4/12/21: 51% Lab Leak
We’ve made two forecasts to the origins of COVID-19 and made substantial updates to both in the past week.
First Question
The first forecast we changed was for this question: Credible claim by 2024 that COVID-19 likely originated in a lab?
We made two key changes to this forecast this week. The first change was in response to the WHO-led team’s report which was questioned by many countries and other significant actors (such as the WHO chief) and asking for a further investigation which would also cover the lab-leak hypothesis.
We reacted to this signal in two steps. The first was to remove the 51-49 weighting effect towards Zoonotic Origins we established due to the WHO report in the previous update in volume 7. The second was to give a 52-48 weighting towards the Lab-Leak hypothesis due to the larger-than-expected negative response to the WHO-report which called for investigations into the hypothesis. This leads to an absolute forecast of a 51% probability of a lab-leak origin for COVID-19.
The second change in our forecast was to reduce the political suppression factor we established in our initial forecast of 50%. As we noted in the original forecast, one thing that would cause us to change our mind was the geopolitical relations with China. Although little has materially changed since we made that forecast, we believe we were far too generous with the 50% discount given the current geopolitical climate. As a result, we believe slashing the 50% reduction to 25% makes more sense.
This leads us to a 38.25% likelihood that there is a credible claim by two major public health agencies that COVID-19 likely originated in a Chinese virology lab.
- There is still some debate between us whether a 25% reduction is still too large, so if you have any thoughts please let us know!
Second Question
The second forecast we changed was the stupid-resolution COVID-origins question, which we lowered from a 25% likelihood down to a 1% chance because we want to maximize our points and it appears highly unlikely that the median forecast will rise to 97% within the next month.
Andrew Eaddy
4/12/21: 82% Lab Leak
For our second update (and spoiler alert: recast) this week, we return to the origins of SARS-CoV-2 and our forecast on whether the lab leak hypothesis is true. First forecasted in volume 6 with a 19% likelihood, we have updated this prediction 2 times so far and it has had one of the most consistent and aggressive changes of all our forecasts.
Why are we revisiting this forecast now? First, the Wall Street Journal recently reported that three researchers from the Wuhan Institute of Virology were hospitalized due to a sickness in November 2019. Second, Dr. Fauci recently said he was “not convinced” of a natural origin for COVID-19—a shift in his previous stance on the potential of a lab-based origin. His shift is one among many, and is a noticeable development. Furthermore, in light of recent acceptance and coverage of the lab leak theory there has been no substantial rebuttal. This is important because it shows a robustness to the theory.
Taken together, these developments have changed our forecast in two key ways.
- We have increased confidence in Rootclaim’s analysis on the origins of COVID; specifically, we increased our confidence on the evidence of chimeric structure and Furin cleavage from 80% to 85%.
- We decreased the penalty given by Rootclaim due to missing evidence from the WIV from 1/2x to 2/3x.
We also added Rootclaim’s additional analysis on COVID’s initial adaptability to our forecast.
At the same time, we are still unsure about the political and obstruction factors for positive resolution. First of all, China could continue to prevent any real investigation into the origins of COVID meaning scientists may be unable to gather enough evidence in the meantime to conclude with sufficient certainty in a lab origin. Moreover, there is still the risk that the public health agencies listed for positive resolution would refuse to endorse a lab-based origin due to geopolitical reasons. Therefore, we are still discounting our true forecast by 30%.
As a result we forecast a 58% likelihood that at least two major public health agencies claim it is more likely than not COVID-19 originated from a Chinese virology lab. (82% likelihood without discount factor.)
Clay Graubard
Metaculus Community Updates
Before getting to our final prediction, here are a few Metaculus community updates!
Community Guidelines: For the first time, Metaculus now has a codified set of community guidelines. While this seems fairly humdrum on the surface, we found this to be an important step for the growing platform. As any online social community hits certain thresholds with respect to scale and notoriety, the introduction of things like community guidelines become necessary. So we see this as an indication that the Metaculus community is healthy and growing which is exciting for the forecasting community writ-large.
Moderator Elections: What good are community guidelines if there isn’t someone to arbitrate on them? Metaculus is in the midst of holding an election process for community moderators. This, like the creation of the community guidelines, is a positive indication of Metaculus’ health, and will ostensibly only help the platform improve moving forward, which we at Global Guessing are all for! If you are interested in applying to become a moderator, or just want to see if you’re eligible, check out the criteria and description here.
Will There Be Peace in Yemen in 2021?
For our second and final forecast in this week’s Metaculus Monday, we are predicting whether there will be a 30-day period in 2021 when either a cease-fire or peace agreement is in place in over 90% of the territory in Yemen. The community median is 52%, while the mean is 50%. So far there have only been 15 predictions: One of our earliest yet!
But before we get into the forecast, it’s important to have an understanding of the geopolitical events which have led to the current moment in Yemen.
Background: Yemen Civil War
In 2011 the Arab Spring led to regime changes in several Middle Eastern countries, Yemen included. Then President Ali Abdullah Saleh was ousted on claims of corruption and rising unemployment, and his Vice President Abdrabbuh Mansur Hadi came to power. This transition of power was not successful, however, and has played a key role in Yemen’s current crises.
Under Hadi, unemployment continued to rise unmitigated. Hadi also led Yemen’s move in 2014 to be divided into six discrete federal regions. While some viewed this as a welcome change to the status quo of violence and poverty in Yemen, potentially offering a more responsive government option, groups like the Houthis were outraged. They viewed this division as an attack on their influence in the country.
The Houthis, a Shia rebel group who existed in the pre-Hadi era, ultimately partnered with former President Saleh to gain greater political influence and troops. The Houthis went on to capture Yemen’s capital Sana’a, and marched on to threaten Hadi’s rule in the coastal city of Aden. It was this move toward Aden which saw Hadi flee to Saudi Arabia (where he currently resides) and in turn which rang alarm bells for Saudi Arabia that something must be done.
And so in 2015 violence in Yemen escalated, with Saudi Arabia leading a massive coalition of countries to fight off the Houthi rebels, whose Shia identity made them a likely ally of Iran, one of the few predominantly-Shia countries in the Middle East. Carpet bombing ensued, with thousands of Yemeni civilians being killed, and even more leaving the country catalyzing what is arguably the worst ongoing humanitarian crisis in the world.
To make matters worse, violent non-state actors have leveraged the chaos in Yemen to gain footholds on the borders, making conflict in the region more complicated. Additionally, some members of the Saudi-led coalition in Yemen have begun to work towards their own goals in the country as the conflict has gone on, weakening the efficacy of the group. So now we are at a point where Yemen has become the violent playground for several countries to assert their interests all while civilians continue to starve and be killed.
Analysis
To start, we tried to identify a base rate for ceasefires agreements since the civil war began in 2014. In the war’s six-year span, there have been two instances of ceasefires. One in 2015 and another in 2020, leading to a 33.3% base rate. However, neither of these ceasefires lasted more than a week. As a result, we cut the base-rate for a 30-day ceasefire by 10x, resulting in a 3.33% base-rate for a 30-day ceasefire.
Next, to get outside of the binary of ‘continued fighting’ and ‘ceasefire,’ we attached a 5% possibility of a peace deal or Houthi victory to take place. We then subtracted the peace and ceasefire possibilities from 100% to get the continued fighting base-rate. The resulting base rates were:
- 91.67% Likelihood of ‘Continued Fighting’
- 3.33% Likelihood of ‘Ceasefire’
- 5.00% Likelihood of ‘Peace Deal or Houthi Victory’
We then took the same approach as the Rootclaim analysis, using our own evidence and key factors:
- Biden ending support for the Saudi-led coalition
- The 2020 prisoner exchange
- Covid-19 pandemic / vaccines
- Recent Envoy developments / Other US Actions
The recent news that Biden is ending US support for the Saudi-led coalition in Yemen is important for two reasons. On one hand, it has led to clear demonstrations of power by the Houthis ostensibly raising the likelihood of a ‘Houthi Victory’ or peace deal struck due to their strong position. It also, however, indicates potential ‘Continued Fighting’ with the violence becoming protracted on the back of a Houthi resurgence thanks to less United States influence. We have an 80% confidence that US disengagement from the conflict made the likelihoods for continued fighting and peace deal / Houthi victory far likelier, giving them a 5x multiple and creating the new probabilities of:
- 94.18% Likelihood of ‘Continued Fighting’
- 0.68% Likelihood of ‘Ceasefire’
- 5.14% Likelihood of ‘Peace Deal or Houthi Victory’
Our second factor is the recent prisoner exchange and UN involvement in the crisis. Last fall, in October, the largest prisoner swap in the six-year conflict took place. Both rebel fighters and pro-Yemeni government affiliates were released to their respective sides, with another exchange already in the works at the time of the swap, showing massive amounts of collaboration between the two warring sides.
The success of the swap increases the chances that a ‘Peace Deal’ will occur, or also potentially a ‘Ceasefire’. To calculate the impact of the prisoner swap on our outcomes, we gave it a double 51-49 weighing, in favor of both a peace deal / Houthi victory and a Ceasefire, leading to the following values:
- 93.72% Keep Fighting
- 0.74% Ceasefire
- 5.54% Peace Deal / Houthi Victory
The next factor is the one most in favor of a ceasefire taking place this year: Covid-19. Covid-19 served as the backdrop for last year’s ceasefire. We also believe Covid-19 could potentially serve as a peace-generating factor as opposed to a peace-limiting factor. Getting past the pandemic is a desired goal for many groups that span ethnic and political bounds. As Yemen just received vaccines in March, this issue is exceedingly relevant for the Yemeni people of various creeds making cooperation to achieve herd immunity much more plausible. Given this analysis, we gave a 0.8x multiple on continued fighting, a 5x multiple on a ceasefire, and a 1.1x multiple on a peace deal / Houthi victory. The resulting percentages were:
- 88.46% Likelihood of ‘Continued Fighting’
- 4.35% Likelihood of ‘Ceasefire’
- 7.19% Likelihood of ‘Peace Deal or Houthi Victory’
Finally we found that both the recent envoy news–that the US’ proposed ceasefire was rejected by the Houthi’s–and U.S. Secretary of State Blinken’s assertion that there can be no military solution to the conflict in Yemen signals a high likelihood of continued violence in the region. The Houthis’ rejection following US-withdrawal from the conflict could indicate a confidence in their strong position further increasing the odds of continued fighting. Given our uncertainty of the final effect these recent developments will have, we give a 51% weight to continued fighting, 49% to ceasefire, and 50% to peace deal / Houthi victory. Given this, we came to the following final values:
- 88.74% Likelihood of ‘Continued Fighting’
- 4.19% Likelihood of ‘Ceasefire’
- 7.07% Likelihood of ‘Peace Deal or Houthi Victory’
Initial Forecast
Taken together, the four factors’ influence on the three potential outcomes of this conflict by 2021 given the question criteria yields a 11.26% likelihood that there is peace in Yemen this year.
What would make us reconsider?
There are also a number of factors that have the potential to affect this outcome which we will be watching over the next 9 months, and more pertinently over the next four months before this question closes on Metaculus.
We have broken those factors into two categories. The first, the endogenous factors, are those which are already baked into our initial prediction. These will be changes of values already present in our model, and so if any of them do sway the question’s resolution, we should be able to understand that outcome with our analysis. The second group are the exogenous factors–those factors which exist outside of our model’s considerations but which still have a significant potential impact on the outcome of the question. While the relevant importance of each factor varies, each is significant enough to affect the outcome and should be considered when forecasting this question. They are as follows:
Endogenous Factors (Part of original prediction)
- COVID Vaccine News in Yemen
- Progress of the UN mission
- Whether Houth’s continue to dominate on the battleground
Exogenous Factors
- Saudi Arabia’s position on the blockade
- Outcome of Iranian election (note: Iran has influence, not control over Houthis in Yemen)
- JCPOA Talks (quasi-endogenous)
- Saudi Arabia’s reconsideration of spending in Yemen (They are spending billions while Iran is spending only 10s of millions)
As this question does not close until mid-summer, we will be watching how both our central considerations, and our endogenous and exogenous factors behave over the coming months. Should there be any big news in any one factor or consideration we plan to update this forecast accordingly. Additionally there are only 15 predictions on this Metaculus question at the time that this article is being published, so we hope that this article encourages more people to make predictions on that question.
3/22/21: 10% Peace
- GG: 11%
- Community: 45%
- Close Date: 7/15/21
- Volume: 6
| New / Revisited Evidence | Effect |
|---|---|
| Houthi drone strike on Saudi oil refinery + Saudis lead retaliatory airstrike on Yemen's capital | Weak: 51%-49% Weight Towards Continue Fighting |
| Houthi major territorial advances | Moderate: 55% Continued Fighting, 55% Peace / Houthi Victory, 45% Ceasefire |
In recent weeks, events have occurred surrounding the Yemeni-Saudi conflict that have had the potential to affect our likelihood that Yemen sees a 30-day period of peace in 2021. On the Yemen-side of the conflict, Yemeni Houthi rebels claimed responsibility for a drone strike on an oil refinery in the Saudi capital of Riyadh. The attack consisted of six drones and hit facilities belonging to Saudi Aramco, one of the largest companies (by revenue) in the world. The Houthis have also made territorial advances in Yemen, seizing boht city and mountain territory previously held by the Yemeni government.
On the Saudi side of the conflict, an attack was carried out in the Yemen capital of Sana’a in retaliation for the oil refinery strike, hitting key Houthi military bases as well as a Yemeni port, hurting the country’s economy. The recent escalation in violence between Yemen and Saudi Arabia is thought to be a potential hindrance to any peace deal in the near future. Specifically, this escalation might result in a tit-for-tat series of attacks from both countries, a strategy derived from mathematical psychologist Anatol Rapoport in one of political scientist Robert Axelrod’s game theory tournaments in the 1970s and 1980s. If you have an interest in political science, political theory, or game theory, definitely look into those tournaments!
Given our sensitivity towards moving percentages below 10% or over 90%, we want to be more moderate when examining the recent news. We find the air- and drone-strike news to be more likely noise rather than signal. Conflict between Yemen and Saudi Arabia has persisted at a high level since 2015, making this news, while potentially relevant to our forecast, not novel in any way. On the other hand, we do not want to completely discount the effect of the Houthis recent territorial gain.
As a result, we slightly decrease our likelihood of peace in Yemen in 2021 from an initial forecast of 11% to a 10% probability.
Clay Graubard5/3/21: 9% Peace
Our final forecast this week is on a previous forecast we made on Metaculus Mondays on whether there will be peace in the Yemen Civil War in 2021.
This question resolves positively if there is a cease-fire or peace agreement in the Yemeni Civil War by January 1, 2022. The agreement will have to last without unambiguous violations for 30 days and cover over 90% of territory in Yemen.
There are currently 35 predictions with a community median of 41% and mean of 35%. When we originally made the forecast, the community median was 45%, and when we updated our forecast (from 11% to 10%) a week after making it the median was 41%.
The primary factors that we considered when updating this forecast were the time constraints for the question (how much time is left in the year relative to our previous forecast update) and any relevant news signals that might inform our thinking.
Whereas in the past, a report about U.S.-Saudi talks to find a solution to the violence in Yemen may have affected our view on the situation more heavily. But other reports, such as Yemen’s recent on a Saudi airbase in Asir, have tempered that. We also weighed the possibility that even if the United States was able to broker talks between the Yemeni government and Saudi Arabia for a ceasefire, the presence of violent non-state actors participating in this conflict may make those conversations moot.
Iran is also a relevant player in this equation, as they have been reported to be backing the rebels currently embattled with Yemeni forces. And with Iranian elections, scheduled for this summer, quickly approaching, we think it is possible that Rouhani may be too preoccupied to effectively stop the country’s support for the Houthi rebels.
Iran is currently in talks with the United States over the JCPOA, and a deal Rouhani would like to get over the line before his re-election campaign. So peace in Yemen would ostensibly be pushed to after the election and the fuller integration of the next Iranian president, making the time constraint for this question even more potent.
Updated Forecast
Taking all of this into account, including the positive and negative effects of Yemen’s geopolitical situation as well as the uncertainty that accompanies the upcoming Iranian election, and the passage of time, we drop our forecast of Peace in Yemen in 2021 from 10% to 9%.
This is more a product of the time that’s elapsed since our previous forecast update than a reflection of any signals that we’ve identified in the interim. We will, however, be closely watching the many moving parts in the Middle East in the coming months, including the Iranian election, JCPOA discussions, Israeli elections, and Saudi-Israeli normalization talks to inform our next update for this question.
Clay Graubard
Finally, I turn our attention to the ongoing civil war in Yemen where fighting has been taking place for nearly seven years with over 200,000 dead. While Saudi air raids have been halted in the recent past to boost the peace efforts, this otherwise signal is overshadowed by the foundation of the peace talks. As pointed out by a recent Foreign Policy article, the terms of the peace treaty proposed by both the US and Saudi Arabia are unfavorable to the Houthis despite their clear dominance and victory in the civil war. The result is that fighting is likely to continue unless either the US and Saudi Arabia rapidly change their approach to the peace process or the Houthis outright win the civil war by the end of the year.
6/21/21: 11% Peace
Will there be a cease-fire or peace agreement in the Yemeni Civil War by 2022-01-01? | Updated: 11% ?+2 (Metaculus: 27%)
Our Take: These conflicting narratives have created two different forecasts for whether there will be peace or a ceasefire in Yemen this year. One lowers the forecast to 8%, the other raises it to 14%. For now, the two forecasts will be averaged but if you have insight into which narrative is more likely to win please let me know in the comments.
Negative Signals
Washington Has Yemen Policy Backward: The Houthis have defeated Saudi Arabia—and peace won’t come by dictating terms to the victors | Foreign Policy 6/3/21 by Annelle Sheline
Mixed Signals
Yemeni officials say battle for key city intensifies | Associated Press via Yahoo! News 6/20/21
Positive Signals
Yemen air raids halted to boost peace efforts: Saudi-led forces | Al Jazeera 6/10/21
Clay Graubard
Global Guessing Loves Geopolitics!
Here at Global Guessing we love geopolitics. That’s made pretty clear from our last forecast of this Metaculus Mondays volume which takes into account a massive about of knowledge regarding bilateral relations and historical ties between countries. It’s also very timely and important to know about as it is affecting hundred of thousands of people each day.
If you want to hear more of our thoughts on the current geopolitical landscape check out our most recent Global Guessing Weekly Podcast episode. We spoke with Ross C. of AR Global Security all about China, Russia, the evolving form of warfare, and more interesting topics.
And subscribe to his YouTube channel for more interesting content and further collaborations!
