Frequently Asked Questions

RFI Overview

What is RFI?

The RAND Forecasting Initiative (RFI) is a new forecasting capability that aims to support policy analysis and decision making with crowdsourced predictions from RAND analysts and an external global community of forecasters. This collective intelligence generates predictive trends and early signals that augment traditional RAND research tools and expertise. 

RFI is supported by Cultivate Labs.

Who do I contact if I need help?

For general questions about the project, click the "Contact Support" button below or send an email to [email protected].

What am I being asked to do as a participant?

We are asking participants to estimate the probability that various events related to geopolitics will occur and to offer a rationale for those forecasts. The purpose of RFI is to generate real-time crowd forecasts, so we ask that you log in regularly (several times per month) to submit new forecasts and make updates. We release new questions periodically and encourage you to check in when alerted of a new set of questions. You may also support RFI in other ways, such as by inviting friends and colleagues to join the community or following our blog and other outputs.

How are the questions on RFI selected?

RFI receives requests from RAND researchers and the policymakers they support about issues or topics where they can benefit from collective predictive signals. We then scope the question that best represents what they want to know, and our team goes through a process of “decomposing” that broad question into the individual questions you see on RFI. Learn more about our decomposition process.

Can I invite my friends or colleagues to become forecasters?

Absolutely. We encourage anyone with an interest in forecasting, science and technology, or geopolitics to apply to join the RFI community. No specific background in any of the topic areas is necessary.

Back to Top

Making and Updating Forecasts

How does forecasting work?

To make a forecast on RFI, simply enter the probability (i.e., 0-100%) you think an event will occur, along with a rationale for your forecast. Here is an article with tips on how to approach your first forecast.

How do you generate the crowd forecast?

The crowd forecast is an aggregation of all individual forecasts at any given time. Initially, the crowd forecast will simply be the mean of all individual forecasts. Over time, we will transition to more sophisticated aggregation methods, such as putting more weight on the forecasts of individuals with the best track record.

Who determines what the correct answer is?

The background section for each question provides information about how we judge the correct answer. In many cases, we simply cite an authoritative data source, such as a well-known database. In other cases, we provide details about how we would evaluate borderline cases. But ultimately, the RFI Team has sole discretion to decide what the correct answer is.

How often should I submit forecasts and how much time should I spend each time I submit a new forecast?

Each participant should submit and update forecasts several times per month, though we encourage more active engagement. Participants should expect to spend several minutes every week on the site reading through new questions and updating forecasts. Updating is highly encouraged: the most accurate forecasters tend to update their predictions more often than average.

What should I include in my forecasting rationales?

Rationales should be short explanations that provide context to others about why you forecasted a certain way. You may share considerations that significantly affected your forecast, as well as links to any sources you found useful and that may be helpful to others. 

We consider a forecast rationale to be of high-quality for the purposes of RFI when it includes a clear, succinct summary of the forecaster's perspective indicating some level of external research. If you choose to cite research, be sure you are summarizing and stay on topic. Please refrain from pasting in lengthy or irrelevant excerpts from sources (e.g., news articles, AI tools, etc) as that defeats the intent of this community. See our policy for using AI tools like Chat-GPT in forecasts to learn more about this topic. 

Can I withdraw my forecast from a question?

There is no way to delete a forecast, but you can submit another forecast to reflect how you've changed your mind. If you forecast more than once in one day, only your last forecast of the day will count toward your accuracy score. You may also quit a question so that your forecasts will only be scored up until the day you quit. You will not receive scores for the days after you quit the question. To quit a question, visit the question you want to quit, click the gear dropdown in the upper left, and click "Quit Question."

I have a specific question regarding a forecast question, how can I get more specifics before I make a forecast?

If you need any clarifications, first read the Information box underneath the question. If you still need clarification, you can submit a New Clarification request directly from the settings within the question page. If you have any issues submitting a clarification request, send an email to [email protected].

What is RFI’s position if I encounter a question where I believe I need access to paid data to make an informed forecast?

All forecast questions on RFI can be answered using open-source information to come up with your best-estimate probability forecast. Where possible, RFI will provide historical or other relevant data within a question’s background section. If additional data is not readily available to you (e.g. it is behind a paywall), we ask that you make your best judgment using what is available. All questions released on RFI have been determined by our team to be important for generating signals for our research priorities. However, you may choose not to forecast in a question in which you feel you cannot make an informed judgment.

Back to Top

Forecasting Accuracy

What does it mean to resolve a forecast question?

When we know the correct answer to a question (e.g., when an event occurs), we can "resolve" the question. An RFI site Administrator will resolve a question, which then allows the site to measure and provide accuracy scores.

How are my forecasts scored for accuracy?

The RFI site uses a measure for accuracy that is also used by weather forecasters, called the Brier Score (the lower your score, the better). This article explains how we calculate the Brier Score.

To measure how accurate you are compared to the crowd, RFI uses an Accuracy Score (or Relative Brier Score). This score is based on the Brier Score, and is generally what shows up on the site leaderboards. This article explains how we calculate Relative Brier Scores.

How can I know my accuracy score?

You can find your cumulative Accuracy Score (or Relative Brier Score) listed in My Profile or on any of the site’s leaderboards. Please note that RFI forecasters will not have Accuracy Scores until a forecast question has been resolved.

Back to Top

Teams

How do I create a team?

• Click the Create a Team link under “My Team” on the navigation menu. If you are already a member of a team,  there will be an option to Switch Teams or Create a new team.
  
• Enter a team name and the RFI usernames or email addresses (if they are not yet signed up for RFI) of all team members you wish to invite.
• Submit your team member invitations – and remember, you can always add more teammates later.
• You can view the invitation status of each team member once they are invited. Be sure to personally follow up with any team members who have not yet accepted your invitation to join the team.

Once you have an approved team, you will be able to interact with members of your team in different areas of the site, for example:

• Go to “My team” on the top navigation menu to view and add other users on your team, their activity, and start general discussions.
• Within any Question page (below the forecasting interface), navigate to the “My team” tab to start a discussion between only other members of your team and compare your team’s aggregate consensus to the crowd forecast.
• Click on Leaderboards in the top navigation menu to see team rankings, which are updated automatically anytime a question is scored.
  

Back to Top

Forecaster Perks & Benefits

I hear that RFI offers rewards from time to time. How can I earn rewards?

RFI offers periodic participation challenges and monetary rewards for active forecasters. Past reward challenges have included forecasting on specific questions and topics and forecasting several times within a specified period. Check out all of RFI’s active reward challenges here.

What is the Pro Forecaster Program? And, how can I become a Pro?

RFI Pros are invited to participate in the program for having a high performing track record of forecasting over a period of time. Pros earn a monthly stipend to dedicate time to forecasting every month. They also participate in training, collaborate on smaller subteams with other Pros, and are invited to exclusive events with topic experts. To learn more about the program, please visit the Pro Forecaster overview page.

What is the Team Lead Program? And, how can I become a Team Lead?

Team Leads help to build the RFI community and diversify perspectives by recruiting and engaging a team of at least 10 active forecasters each month. Team Leads are compensated each month for engaging their team and ensuring member participation. To learn more about the role and how to apply, please visit the Team Lead overview page. 

Does RFI offer forecasting training?

RFI is working towards having training opportunities for its RAND researchers and other forecasters in this community. More information will be provided at a later date. 

Back to Top