The need for "Considered Estimation" versus "Conservative Estimation" when ranking or comparing predictors of job performance.

A recent attempt to generate an updated ranking for the operational validity of 25 selection procedures, using a process labeled "conservative estimation" (Sackett et al., 2022), is flawed and misleading. When conservative estimation's treatment of range restriction (RR) is used, it is unclear if reported validity differences among predictors reflect (i) true differences, (ii) differential degrees of RR (different u values), (iii) differential correction for RR (no RR correction vs. RR correction), or (iv) some combination of these factors. We demonstrate that this creates bias and introduces confounds when ranking (or comparing) selection procedures. Second, the list of selection procedures being directly compared includes both predictor methods and predictor constructs, in spite of the substantial effect construct saturation has on validity estimates (e.g., Arthur & Villado, 2008). This causes additional confounds that cloud comparative interpretations. Based on these, and other, concerns we outline an alternative, "considered estimation" strategy when comparing predictors of job performance. Basic tenets include using RR corrections in the same manner for all predictors, parsing validities of selection methods by constructs, applying the logic beyond validities (e.g., ds), thoughtful reconsideration of prior meta‐analyses, considering sensitivity analyses, and accounting for nonindependence across studies. Practitioner points: A recent, updated ranking of the validity of selection procedures uses a fundamentally flawed and misleading procedure labeled "conservative estimation."The procedure is logically flawed because when validities are directly compared, some values are corrected for range restriction while others are not. We empirically simulate how this creates bias and leads to incorrect decisions.The list of selection procedures includes both predictor constructs (e.g., cognitive ability tests) and predictor methods (e.g., interviews), and we reaffirm how construct saturation within methods causes additional confounds that cloud comparative interpretations in both validities and sub‐group differences.As an alternative, we suggest and outline a "considered estimation" strategy. Basic tenets include using thoughtful range restriction corrections across all predictors, parsing validities of selection methods by constructs, conducting sensitivity analyses, and using a full array of available data (as demonstrated in our reanalysis of general mental ability). [ABSTRACT FROM AUTHOR]