Research scientists, analysts, and other technical experts are critical in informing policy decisions for a variety of high-impact national and global risks. Scientists are often the first to uncover and report risks long before they are recognized as actual risks, such as discovering a novel virus, identifying new technology that could be used in warfare by adversaries, or detecting patterns indicative of future environmental crises.

However, assessing and acting quickly on long-term scientific risks can be challenging for policymakers for a variety of reasons. For example, policymakers may not be equipped with the full picture, including an understanding of the near-term and medium-term milestones associated with a specific risk to engender immediate policy action. Thus, a coordinated response can get delayed or pushed back to the point that the risk becomes an imminent danger.

INFER aims to strengthen the connection between science and policymaking when it comes to assessing these kinds of scientific risks by providing a new source of information that can serve as a complement to scientific assessments. This blog post outlines three ways that INFER strengthens policymaking in this context: (1) aligning scientists and policymakers on the “big picture”, (2) improving the precision and action-ability of long term risks, and (3) increasing the overall accuracy and credibility surrounding risk forecasts.

Aligning scientists and policymakers on the “big picture”

INFER’s process for issue decomposition helps to align policymakers, analysts, and experts on the key factors and signals that can be monitored to understand how a risk will play out. From the decomposition, INFER develops a set of forecast questions that help analysts monitor and be alerted to any movement in the direction of that risk. Understanding the risk through the lens of INFER’s issue decomposition has many benefits, including making otherwise opaque or unclear assumptions underlying risk assessments more transparent and contextualizing the risk within various future scenarios.

Once the set of questions is defined, forecasts and rationales are aggregated from INFER’s crowd of researchers and citizens from inside and outside the U.S. Government to identify any areas of consensus and disagreement.

For example, INFER worked with stakeholders on the issue of international competitiveness in artificial intelligence, decomposing it into key factors, including academic research, human capital, and technological innovation. The decomposition allowed for a shared understanding of the factors that impacted the U.S.’ ability to lead in AI, and contextualized the aggregate crowd forecasts in a way that could supplement existing scientific analysis on risks to AI competitiveness.

Improving precision and action-ability

Policymakers are frequently inundated with scientific literature on potential risks, which can make it difficult to ascertain a general consensus on the likelihood of events and determine the path forward to mitigate the risk. By aggregating forecasts from a diverse community, INFER creates a synthesis of collective knowledge. Forecasters submit their probabilistic assessments based on their own experience, knowledge, and independent research. They are also submitting forecasts on an ongoing basis, meaning that any new information is accounted for in their updated assessments. Analysts using INFER to report to policymakers on rapidly moving situations can therefore be confident that the forecast data will be up-to-date.

Here’s an example: The aggregated INFER forecast might tell you there’s a 5% chance of a Russian nuclear strike on Ukraine within the next 100 days. This level of precision, combined with the forecast aggregation mechanism, is what makes INFER unique and valuable. By incorporating the views of many scientists, analysts, and researchers (and updating based on how those views change over time) the INFER forecast becomes an asset in implementing effective policy action and building mutual understanding among scientists and policymakers.

Increasing the overall accuracy and credibility surrounding risk forecasts

Capturing forecasts from a diverse crowd has been shown to be far more accurate in evaluating risks, including risks from infectious disease, than relying on any one expert’s perspective. Since INFER’s aggregate forecasts are inclusive of the community’s vast knowledge and expertise, this multitude of perspectives reduces collective bias and results in higher accuracy. As forecasting becomes more accurate, policymakers build trust in the crowd. This can help avoid either the “boy who cried wolf” problem (if frequent warnings of high-probability risks are never vindicated) or the problem of “right-tailed” (high magnitude, low probability) risks being ignored entirely when there is no number attached to it.

There are many examples of accurate signaling on INFER. This year, INFER predicted as early as January 30, 2023, a 90%+ chance that Japan would join the U.S. chip ban on China on or before March 31. Japan did join the ban on March 31. INFER also predicted an 89% chance that the Brothers of Italy party would win the most seats in the 2022 general election, which they did. Perhaps even more important than accuracy itself is the scoring and tracking of accuracy over time, which creates transparency about the track record of the crowd and individual experts and lends credibility to this as a valuable source of information for policymakers.

INFER is a powerful mechanism for bridging the gap between science and policy – and by design, the program is informed and transformed by both scientific experts and policymakers. By improving the usability and accuracy of forecasts, and by promoting issue decomposition to increase mutual understanding of the “big picture” outcome, INFER is helping scientists and policymakers communicate and work together more effectively to proactively address critical risks facing our world today.