Geoff Odlum is the President of Odlum Global Strategies and a former Foreign Service Officer within the State Department. Trent Hesslink, is a recently retired Assistant Professor of the National War College and former CAPT, U.S. Navy. We asked Mssrs. Odlum and Hesslink about their experiences in forecasting, and the impact of crowd forecasting for governments and national security. Here’s what they shared.
How did you get started with crowd forecasting? When did you first get involved with the INFER program?
Odlum: When I was still at the State Department, I was aware of superforecasting, but I wasn’t a participant. I didn’t incorporate any of those tools or principles or techniques when I was a diplomat, and that was a huge oversight. After retirement, an old colleague told me about Good Judgement, and I signed up for fun. I enjoyed it. That led me to Georgetown University’s Foretell, the crowd forecasting program which transitioned into INFER.
Hesslink: I was associated with Foretell, and when the program shifted over to INFER, I submitted an application to be a Pro Forecaster. When I joined the National War College, I really started digging into decision models and a little bit of prospect theory. I really got interested in the difference between forecasting, foresight, and futures.
How does forecasting as a capability relate to your career in national service?
Odlum: In my time at the State Department, I served overseas about half of the time, doing political tours in Europe and the Middle East. The rest of the time was in Washington at the State Department headquarters, learning about how policy formulation happens. I worked a lot on nuclear nonproliferation and security with NATO, and counterterrorism in Iraq. What’s been interesting to me is that a number of the INFER questions we have been asked to forecast on have definitely been issues I have worked on.
Hesslink: The National War College wants to teach all students to get comfortable developing security strategies. In that line of work, we have to state the future outcome that you are either trying to achieve or that you’re anticipating you need to respond to. The students have also demonstrated to me that they see forecasting as a key step in their leadership development, the idea that they can dissect their own decision making processes and then help use that in their team building models.
How might you have used crowd forecasting in your prior work as officers in the Foreign Service and the Navy, respectively?
Odlum: Forecasting would’ve helped me use more critical thinking, and more awareness of all the cognitive biases we fall into. I tended to put too much stock in the view of experts. A lot of the assessments we got in the Foreign Service, like in Afghanistan, turned out to be wrong. Challenging your own assumptions is a core practice of being a successful forecaster.
Hesslink: The reason I jumped on forecasting is because I understood it was going to make me a more effective leader and that I would make fewer bad decisions. One of the things that we really emphasize at the National War College is that forecasting is going to demand that you become more open-minded and accept information from the world, and I think that’s a key behavioral trait that people learn from forecasting. The students I have touched have all walked away feeling like ‘I’m going to be a better leader because I’m going to be less biased.’
What benefits do you think crowd forecasting offers to national security?
Odlum: There’s tremendous value that has come from forecasting, because it helps teach anyone how to ask better questions and how to challenge assumptions when analyzing an issue, which would be super important for Foreign Service Officers (FSOs).
The intel community is often too careful and conservative with their forecasts, and instead of coming up with probabilities, they come up with broad confidence assessments. A course for all FSOs to learn about forecasting, base rates, anchoring, checking your own biases, and more would be really important.
Crowd forecasting would also be quite useful in, for example, an Interagency Policy Committee meeting where representatives from different agencies come together to reach an agreement about some recommendation to the president. People from different bureaus come with their own perspective, but it’s all qualitative discussion. So it would be great in one of those meetings to present about forecasting and get some quantitative perspectives in there.
What challenges do you see in implementing crowd forecasting in government and intelligence communities? And what can help remedy those challenges?
Hesslink: In both governmental policy and the field of management there’s a growing trend that you can reduce uncertainty through machine learning and data analysis. What we spend a lot of time on is making flawed decisions because you’re biased, and you know that you’re biased, and that’s something all leaders need to understand. We want to define that line between reducible and irreducible uncertainty… Government leaders are starting to dissect their own estimations.
Also, you generally only see detailed post-mortem analysis in government after an event when it has high consequences, say 9/11. But the key will be introducing this routine regularly without making the team shut down. People may be resistant, but it is important – for forecasting and beyond – to work on a culture of improvement.
Is there anything else you’d like to share about the importance of forecasting for advancing national security efforts and policy making?
Odlum: The CIA is not in the business of giving policy advice. They’re only giving intelligence assessments of the dynamics involved in what’s currently happening. That’s where INFER and forecasting can come in – identifying the probability of a successful outcome of option 1 vs option 2. Crowd forecasting has the power to bridge the gap between intelligence assessments and policy recommendations.