A study published in Frontiers in Psychology suggests that psychology’s famous Cognitive Reflection Test is associated with better job performance and training outcomes. The cognitive test was based on a three-item questionnaire designed to measure a person’s ability to find a difficult answer instead of an easy one.

“[We] present a meta-analysis of the validity of the Cognitive Reflection Test for predicting job performance and training proficiency,” say the authors of the research, led by Inmaculada Otero of the University of Santiago de Compostela in Spain. “The Cognitive Reflection Test proved to be an excellent predictor of job performance and training proficiency.”

For readers unfamiliar with the Cognitive Reflection Test, it asks people to answer three brain-teaser-style questions, shown below. (Answers are provided at the end of the commentary)

1. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?
2. If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
3. In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?

The Cognitive Reflection Test assesses a person’s ability to forgo intuitive yet incorrect responses in favor of correct answers that require a deeper level of thinking. The researchers found that people’s performance on the Cognitive Reflection Test was useful in predicting job performance above and beyond other measures of intelligence, such as an IQ test, an emotional intelligence test, or a work sample test. In other words, the Cognitive Reflection Test taps into a type of intelligence, known to psychologists as “System 2” thinking, that is unnoticeable to other forms of testing.

“Cognitive reflection […] is based on the view that the human mind operates through two types of cognitive processes known as System 1 and System 2,” state the authors. “There is high consensus among researchers that System 1 is fast, automatic, unconscious and impulsive and operates with little (or no) effort, and that System 2 is slow, reflective, and purposeful and requires effort and concentration. Moreover, System 2 requires attention, and it is capable of solving complex problems with a high degree of accuracy.”

Because of this, the study suggests that companies and hiring managers would be wise to include the Cognitive Reflection Test in their hiring toolkit.

“As the Cognitive Reflection Test correlates significantly with job performance, they could be included among the procedures used for personnel selection, particularly when cognitive intelligence tests are not included in the batteries of selection procedures,” says the study.

To arrive at this conclusion, the researchers scoured the research literature for studies that tested the association between the Cognitive Reflection Test, job performance, and training proficiency. They found seven studies that tested the relationship between the Cognitive Reflection Test and job performance and 15 studies that tested the relationship between the Cognitive Reflection Test and job proficiency. In both cases, they found evidence to support the notion that the Cognitive Reflection Test is a valid interpreter of future job success.

Keeping things in perspective, the researchers point out that the Cognitive Reflection Test is merely one of many factors that influence job performance, and that many of the things that contribute to job success have little or nothing to do with cognitive ability. Recent research, for instance, has shown that personality characteristics such as “grit,” or the ability to exhibit passion and perseverance for long-term goals, and mental toughness are also important parts of the success equation.

Moreover, few people get all three answers correct. According to other published research, approximately 40% of people don’t get any of the three questions right and approximately 20% of people get either one, two, or all three questions correct.

Answers to the CRT:

1. Intuitive answer $0.10, correct answer $0.05
2. Intuitive answer 100, correct answer 5
3. Intuitive answer 24, correct answer 47

Ref:

1. Otero, Inmaculada, Salgado, Jesús F. and Moscoso, Silvia: Criterion Validity of Cognitive Reflection for Predicting Job Performance and Training Proficiency: A Meta-Analysis (https://www.frontiersin.org/articles/10.3389/fpsyg.2021.668592/full)
2. PabloBrañas-Garzaa, Praveen Kujal, Balint Lenkeib: Cognitive reflection test: Whom, how, when (Cognitive reflection test: Whom, how, when – ScienceDirect)
3. Mark Travers: These 3 Questions Can Predict Future Job Success, According To New Psychological Research (https://www.ivyexec.com/career-advice/2021/cognitive-reflection-test/?utm_source=Sailthru&utm_medium=email&utm_campaign=B2C%20-%20Insights%20-%20B2C%20Newsletter%20Basic%20-%207%2F7%2F2021&utm_term=Insights%20-%20Regular%20-%20Smartlist%20-%20Basic)