Assessing overconfidence among national security officials

National security officials are “overconfident,” which makes it difficult for them to accurately assess uncertainty, according to a new study by a Dartmouth government professor. According to the study, when they thought the statements had a 90% chance of being true, they were only true 60% of the time.

The results were published in the journal Texas Homeland Security Review.

About 1900 national safety Officials from more than 40 NATO allies and partners were surveyed about the uncertainty of the current and future state of the world and provided a total of 60,000 assessments. The officials were enrolled at the US National War College, the Canadian Forces College, the NATO Defense College and the Norwegian School of Military Intelligence.

In the United States, Canada, and Europe, officers upon reaching the rank of colonel are required to obtain a master's degree from a military college or other military institution as part of their professional military education. The participating agencies also included a significant number of civilian national security officials who work in the ministries of foreign affairs and intelligence servicesamong other areas. Thus, the study included an unusually large and representative group of senior national security officials.

The survey contained questions about international military, political and economic issues, and asked respondents to rate the likelihood that a statement was true, for example: “What do you think are the chances that NATO members will spend more money on defense than the rest of the world combined?”

Other questions included predictions, such as: “What do you think are the chances of Ukraine and Russia officially declaring a ceasefire by a certain date?”

The results showed that national security officials are overconfident about the current and future state of the world. cognitive bias this was the same for all respondents, civilian and military professionals, men and women, and US citizens and non-US citizens. They share these prejudices with the general public.

“National security officials are like many of us, in that we tend to think we know more than we actually do. This means that national security officials, like members of the general public, are chronically overconfident,” says study author Jeffrey Friedman, an associate professor of government and a member of the Davidson Institute for Global Security at Dartmouth. He says the overconfidence among national security officials is similar to the bias he found in similar surveys among undergraduates, graduate students and finance professionals.

“However, the study also showed that you can significantly reduce this bias with just two minutes of training,” says Friedman. His study found that briefly presenting national security officials with data on patterns of overconfidence led study participants to make judgments that significantly reduced overconfidence and increased accuracy.

The study also found that national security officials are biased against false positives– the tendency to think that false statements are true.

This was demonstrated by changing the wording of the survey questions. The surveys asked half of the participants, “Has ISIS killed more civilians in the last decade than Boko Haram?” and the other half were asked: “Has Boko Haram killed more civilians than ISIS?” National security officials' responses to these two questions consistently exceeded 100%.

Friedman says this finding indicates that national security officials appear to have a “tendency to confirm rather than deny the possibilities they are asked to consider,” which could be especially problematic for national security officials given that there could potentially be multiple outcomes in military scenarios rather than just one.

The study did offer one potential remedy for overconfidence—reminding national security officials of the dangers of being overconfident in their beliefs.

Before a random group of national security employees took the survey, they received information about the overconfidence and bias of other cohorts. With this two-minute training and informed approach, participants were significantly better at judging uncertainty.

Friedman says the four military agencies that took part in the study deserve enormous credit for their participation. The work had a ripple effect: the first group was from the National War College, which was so pleased with the session that it invited Friedman back, and then other military institutions joined in, following recommendations from previous participants. “It was very rewarding to see how receptive the national security staff was to the information and training they received, as the training was then incorporated into core training programs across the agencies,” Friedman says.

“Any organization that cares about improving people's ability to more accurately assess uncertainty can provide this training,” says Friedman. “The materials are available online and can be developed and integrated into curricula anywhere, whether military officialsdiplomats, intelligence officers, business leaders and others.”

“By using the tools of decision science, we can improve people's judgment,” Friedman says.