Last month, INFER hosted a kickoff event to welcome the newly selected Pro Forecaster cohort for 2023. The event featured a keynote speaker in the federal government who discussed how crowd forecasting can be used to improve rationality in government decision-making and integrate real world signals to solve public problems.

The kickoff event reinforced the important role that this season’s INFER Pro Forecasters (“Pros”) play in relaying external signals to government stakeholders.

Generating value for government decision-makers

Pros earned their designation this season by demonstrating consistent engagement and high accuracy throughout the past season. Having an established track record of performance is necessary for the role that Pros take on as part of INFER.

Besides consistently submitting forecasts and staying up to date on forecast questions, Pros provide well researched, thoughtful rationales for their forecasts. These qualitative rationales are used in INFER’s reports for government decision-makers to help them better understand the context behind the crowd forecast trends. Rationales help to explain why the crowd forecast may be different from internal government forecasts, or even why there might be disagreement among the crowd itself.

The team aspect of the Pro program is another part of what fuels their collective value. Pros are placed into one big team on INFER, and are also split into smaller sub-teams with question assignments, which allows them to collaborate and keep each other informed of new developments on questions. They probe and test one another’s assumptions and help to call out one another’s biases. This dedication to collaboration within question activity threads helps drive other community rationales and commentaries thereby elevating the quality of the crowds’ assessments.

What’s new for this season’s Pros?

The 85 Pros selected in 2023 for the paid program represent diverse forecasting experience, career paths, and geographic locations. Among the Pros are current and former members of the U.S. Department of State, graduate students in policy and government, scientists, political consultants, and U.S. civil servants. Half of the Pros reside outside of the U.S., together representing 21 countries across four continents. Several Pros have accrued years of forecasting experience and are active Good Judgement Superforecasters, while a few Pros were drawn to forecasting just last season and performed exceptionally well in INFER’s Effective Altruism Tournament.

This season, INFER named 10 Pro All Stars, a new designation for the highest performing Pros from last season in terms of accuracy and engagement. All Stars are considered to be forecasting role models for Pros and the entire community. The All Stars can be recognized by their special badge and receive slightly higher payment as Pros for their earned history of high performance.

Congratulations to the 2023 Pro All Stars:

• Ryan Beck
• Dawna
• Cassandra
• Alex Demarsh (check out his INFER interview in this blog post)
• Kyle Green
• galaga
• sebawi
• michalbod
• Jonathan Mann
• DippySkippy

The 2022 Pro track record

Since its inception in 2022, the Pro program has been a significant growth driver of INFER. Last season, Pros submitted 21,225 forecasts and 24,322 comments. As shown in the charts below, forecasts and comments from Pros represent the bulk of INFER’s activity.

Forecast and comment counts on INFER in the 2022 season, by Pros and Non-Pros

Pros have also proven to be consistently more accurate than the crowd. Three of the top ten most accurate INFER forecasters are Pros, including the highest ranked forecaster, Cassandra. As a team, the Pros had the second highest score over the course of the season, with a relative brier of -1.04.

Another success story we can glean from the Pros is that they were able to improve their accuracy over the course of last season as shown in the chart below. Research, as can be found in Philip Tetlock’s book Superforecasters, tells us that forecasting performance can improve over time, especially through the benefit of teams; the Pros are a testament to how this happens in practice.

Pro Forecaster Accuracy Improvement During 2022 Season

Many Pros have reported that being able to engage in discussions with other Pros is one of the main advantages of being in the program. While Pros are individually highly accurate and engaged forecasters, they also continue to learn from and with one another – improving accuracy and outcomes for the entire INFER community.

Congratulations to the 2023 INFER Pro Forecasters and All Stars! If you are interested in becoming a  Pro, be sure to make frequent and accurate forecasts with detailed rationales. We suggest “following” other Pros (as designated by their ‘23 Pro badges) and engaging with them in the commentaries, and learning from some of their practices. The INFER team selects and invites new Pros every season, with some rolling admissions throughout the season if spots become available. Learn more about the Pro Forecaster role.