Welcome to the 12th volume of Metaculus Mondays. It’s hard to imagine we have been forecasting questions on Metaculus for this series for three months – but here we are!
Before we get into this week’s volume, we would first like to announce that Gaia Dempsey, the CEO of Metaculus, will be making an appearance on the Global Guessing Weekly Podcast later this week. If you have a question for Gaia about forecasting, her start-up background, or Metaculus, leave a comment or reach out to us on social media. We will select the best ones for the rapid-fire round.
Moving to this week's forecasts, in volume 12 we are:
- Forecasting whether Russia annexes Ukrainian territory before 2022 — Community Median: 14%
- Recasting (>5% update) last week’s forecast on whether there will be a renewal of intense fighting in the Donbass this summer — Community Median: 30%
- Updating our forecasts regarding the United States’ return to the JCPOA (60%) and Saudi-Israeli normalization by 2022 (43%)
In addition to forecasting these questions on Metaculus, we will forecast similar questions when they exist on Good Judgement Open.
If you want to forecast these questions before reading, you can find them here (one; two; three and four; and their GJOpen counterparts here: three and four). If you have your own forecasts on these questions, respond to our comments (links after each forecast) on either Metaculus or Good Judgement Open and tell us what we missed or how we made you rethink your forecasts.
Will Russia Annex Ukrainian Territory?
How likely is it that Russia annexes Ukrainian territory before 2022? If you’ve read our past interviews or listened to a featured guest episode of the Global Guessing Weekly Podcast, then you will know we have been considering this possibility for months. We asked 2 Superforecasters, one leading researcher in forecasting, and the founder of Rootclaim whether Russia will annex territory in Eastern Europe over the next 5 years. These were their rapid-casts:
- Regina Joseph (Superforecater): 45% -> (this week) 25%
- Balkan Devlen (Superforecaster): <5%
- Pavel Atanasov (Researcher): 15%
- Saar Wilf (Entrepreneur): 15%
Taken on average, our well-qualified guests produced an aggregate forecast of 15%. Given these forecasts relate to the entirety of Eastern Europe rather than just Ukraine, 15% is too high since there’s a non-zero chance for non-Ukrainian annexation. Reduced by 10% to 13.5% produces a 0.225% monthly likelihood for annexation, leading to a 1.8% likelihood of annexation before 2022.
However, Balkan Devlen wrote yesterday in the National Post that “despite the recent announcement by Russian Defence Minister Sergey Shoygu that some of the troops are returning to their bases, the potential for two simultaneous crises triggered by the Russian and Chinese regimes is real and will be one of the key strategic issues that the West needs to deal with going forward.” If we were to increase his forecast by 2x to account for the sentiment conveyed by Dr. Devlen’s recent article, then we’d reach a new overall likelihood of… 1.95%.
Moreover, the base-rate for the past ten years is 10% which produces a 6.67% likelihood by the end of this year.
Taking an average of these two likelihoods to account for the increased tensions between Russia and Ukraine this year, as well as an additional 1% likelihood of new territory due to appeasement, we forecast a 6% likelihood Russia annexes Ukrainian territory before 2022.
Will War Breakout Between Russia and Ukraine?
This week we are also revisiting last week’s forecast on the situation in the Donbass.
On April 21st, Russian president Vladimir Putin shared in an address that, among other updates in Russian foreign policy, troops would be removed from the border with Ukraine on May 1st. Specifically “commanders of the 58th and 41st divisions, have been instructed to return to their permanent base from Friday and all return to their base by May 1st”.
While our first forecast for this question was 51%, or more likely than not that there would be greater than 250 deaths in any given month in 2021, our forecast dropped massively after Putin’s address. On Twitter, upon the news of his address, we lowered our prediction to 15%.
'Russian troops to pull back from Ukrainian border' @POLITICOEurope
— Global Guessing (@GlobalGuessing) April 22, 2021
Forecast: Will there be a renewal of intense fighting in the Donbass this summer?
Original: 51%
New: 15% ?https://t.co/UqNcEAd4Zn
We have now come to understand that the announcement of troop withdrawal is different then the actual removal of troops from the border with Ukraine. Moreover, Russia will be leaving much military equipment nearby the Ukrainian border, meaning a buildup could happen again in short notice. Finally, in 2014 violence picked up even as troops were removed from the border, making that Putin’s statements not necessarily diagnostic to the question at hand.
The bottom-line synthesis of these factors is that the 15% was too low of a drop initially. That being said, a lot of time has passed since the first forecast of 50% and much of the insights Aaron Schwartzbaum shared last week (including the build up of the Ukrainian military) are still in play between the two countries.
Weighing all of the things together, we have raised our prediction number from 15% to 20%.
The factors we are watching has mostly remained the same, except that the biggest development we'll be tracking is whether or not Russia sticks to the May 1st deadline and how the troop placements and skirmishes develop afterwards. We discussed this on the most recent episode the Global Guessing Weekly Podcast!
Clay Graubard
Friend of the podcast and Superforecaster Regina Joseph also reached out to us, sharing that activity in the Sea of Azov will also produce signals for this forecast. An offshoot of the Black Sea and touching Ukraine and Russia, the body of water has immense geopolitical significance. Moves against ports in that region would be a signal of Russian intentions with Ukraine given how potentially destabilizing the move would be, and we will be following that space as well
- Regina is participating in a talk on Tuesday (4/27) with the CTO of the RIAA for WHO KNEW: The Smartest People in the Room. Both of us will be attending, so if you are interested in listening to how a Superforecaster thinks you should sign up here.
Will the US Rejoin the Iran Nuclear Deal?
Over the past few weeks the United States and Iran have engaged in talks to bring Iran back into compliance with the nuclear requirements of various world powers. In January Iran was reported to be enriching uranium at higher concerning rates as a response to crippling economic sanctions placed on the Shia-majority country by the former Trump administration. These talks have been central to our thinking around the question of whether the United States will rejoin the JCPOA by 2022.
Our initial forecast in mid-February was that there is a 20% chance that the United States either lifts or waives sanctions on Iran ‘greater or equal to those mandated under the original JCPOA’ or negotiates a new deal with Iran. At that time, the community prediction on Metaculus was 55%.
In the interim period that number dropped to as low as 35% on April 1st, but has picked back up recently. As of today, the current Metaculus community prediction is 60% with 222 predictions registered so far. And in a similar range to the Metaculus community, the current forecast on the Good Judgement Project is 54%.
For this question the Global Guessing prediction is far off from both of the aforementioned platforms. This is in large part due to things we've learned from past forecasts, namely our war in the Donbass prediction, regarding news announcements and signals. Putin's announcement of troop withdrawal from the border is not the same as evidence of troops being removed, so announcements of actions are not always diagnostic (a signal) to a question.
During the talks between Iran and the United States there have been reports about how talks have progressed that likely have affected people’s views on this question. Many of these reports, however, speak to the preference of Iranian and United States leaders, whereas in our view the constraints which we originally identified are still in place.
We also believe that news reports about updates in the process are not necessarily diagnostic and we have raised the bar for what a true step towards a deal would mean between these two nations. Additionally, the progress being made in these talks does not necessarily mean that the Metaculus question, which has the specific criteria mentioned earlier, will resolve positively.
Finally, time has also passed since our initial forecast, adding downward pressure to our forecast. Taking that into account, as well as how talks have progressed, and the mixed pressure effects of the upcoming election, our updated recast on this question is 20%. This is a 40% increase from our initial prediction of 14% but keeps our prediction in the same ballpark with respect to our forecasting terminology.
Moving forward we will be watching the Iranian election, as a forthcoming election may put pressure on Rouhani to get a deal done to boost his chances. On the other hand, the election may result in a situation similar to what happened to former President Obama when he tried to get Merrick Garland approved by Congress to the Supreme Court – domestic constraints may intentionally stymie a deal. This update is inline directionally with moves on the other two platforms mentioned earlier, but with respect to magnitude we are being much more conservative regarding the current situation and the updates coming out about talks.
Will Israel and Saudi Arabia Normalize Relations?
This week we are also updating our forecast on the question of whether Saudi Arabia will normalize relations with Israel before the end of 2021. The median prediction for this question has changed since our initial forecast on both platforms that we track, and our Global Guessing prediction has changed as well.
The median prediction on Metaculus is now 43%, down from 45% at our initial forecast in February of 21%. At Good Judgement Open, the community forecast is 4%.
In our initial forecast some of the factors we were considering were:
- Saudi Arabia’s leadership dynamics
- The situation in Palestine
- Saudi government constraints
- Biden’s political priorities
While there hasn’t necessarily been a change in any of those factors since our initial prediction, there are two other factors that we considered that are the impetus behind our update.
First, two months have passed since our initial prediction which lowered the odds for this question slightly. The downward pressure effects of this factor are only slight because recent months have been occupied with an election and COVID vaccinations, making the time remaining slightly more irrelevant. However, ten months versus eight months is relatively notable. We measured the effect of this factor as ~2% or a 10% reduction.
Next, we considered a recent report that Saudi Foreign Minister Faisal bin Farhan said normalizing ties with Israel would bring ‘tremendous benefits’ to the Middle East. At the same time, however, the foreign minister also said that any change in relations would be predicated on a shift in the Israeli-Palestine situation. We considered this to be a minor signal towards normalization.
One potential read of this report is that Saudi Arabia’s foreign policy decision maker is trying to force Israel to make concessions towards Palestine. The reason for this would be that Saudi Arabia would like to normalize relations with Israel, but the Palestinian crisis is a constraint on their ability to do so. In our initial forecast, we said that while both countries have a desire to normalize relations, constraints such as these make it difficult. Saudi Arabia has opened the door for Israel to lessen this constraint with positive action. However, given a 56% median likelihood on Metaculus and $.53 price on Predict It for a second national election in 2021, this signal is also minor.
We gave the positive effects of this report at 1%, with a net effect of both new factors at -1%. This puts our updated forecast at 20% chance that Israel and Saudi Arabia normalize relations before 2022.
Moving forward, we’ll be watching whether the possibility of another Israeli election materializes. We’re also keeping an eye on statements and actions of Saudi Arabia with respect to Israel and the Palestinian crisis in particular. However, where things stand today we feel fairly confident that this question will resolve negatively.
