Kurate.org
Rankings
Back to Rankings

Fast, Cheap, and Fair? Evaluating Decision Models

Muhammad Uzair Ul Haq, Davide Rigoni, Alessandro Sperduti, Flavia Fabris

Nov 3, 2026arXiv:2611.04510v1
cs.AI
Share
Scorecard· 16/16
5.5/10 impact

Solid, careful audit with a useful noise-floor framing and released data/code, but narrow data scope, a proprietary model-specific focus, and incremental technique limit broad impact.

Abstract

Industries are increasingly adopting AI-driven systems to automate a wide range of tasks, including high-stakes decisions in human resources, such as candidate screening and recruitment. This makes evaluating their fairness essential, particularly when decisions directly affect individuals' opportunities. Accordingly, the EU AI Act classifies recruitment AI systems as high-risk, imposing strict requirements on their deployment. In this context, compact ``System~1'' decision models offer a fast and cost-efficient alternative to general-purpose LLMs, yet their fairness remains largely unexplored. This article presents the first systematic bias and fairness audit of \jev{}, a typed-decision model targeting recruitment applications. The study evaluates how explicit demographic attributes and implicit candidate cues influence screening scores, rankings, and shortlisting outcomes. These effects are compared against a noise floor derived from name substitutions and typographical variations. The evaluation also examines output validity, latency, and cost. Counterfactual CV datasets and evaluation code are released for reproducibility.

AI Impact Assessments

(1 model)

Impact Assessment: "Fast, Cheap, and Fair? Evaluating Decision Models"

Core Contribution

The paper audits demographic bias in compact "System 1" decision models (JEV, Laya) used for CV screening and compares them with two generative LLMs (GPT-6 Luna, GPT-6.1 Sol). Its main conceptual point is that fairness must be measured against a run-to-run noise floor. Generative models are unstable across identical inputs (Disag@5 of 18–30%), so a single-run counterfactual audit can hide or exaggerate demographic effects. The paper also reports validity (JD-FIT) and efficiency (latency and cost), so the findings amount to a performance/fairness/cost trade-off analysis. The headline findings:

  • JEV is nearly deterministic (ICC 0.999).
  • JEV shows small but statistically significant demographic effects, mostly name-driven. Excl@5 reaches 8.2% against a 2.9% noise floor, and the direction favors women and Black or Hispanic candidates.
  • JEV matches the LLMs on AUC and Precision@5 at roughly 3–4% of the latency and 1.5–18% of the cost.
  • Laya is deterministic but nearly random on validity (AUC 0.52). It also moves almost every CV in rank under any edit.

    • Methodological Rigor

    Strengths

  • The design is a paired counterfactual one (explicit attribute line vs. name-only), with controls on name gender for ethnicity and origin.
  • Seven repeated runs give a null counterfactual.
  • Statistics are careful: paired bootstrap, BH correction, and resampling at the job-description level for clustered data.
  • The sensitivity analysis over classification thresholds shows the model ranking is robust.
  • The rubric is identical across model types, with the same scoring formula, so differences reflect the models rather than the pipeline. It contains no fairness instructions, which is a defensible choice.
  • Thresholds are calibrated on a separate tuning split.

    • Weaknesses

  • Bias is evaluated on a single dataset: 11 job descriptions, 572 CVs, and only two names per group, all generated by an LLM and following US/UK conventions. Name-specific effects are therefore confounded with the particular names chosen.
  • Bias is not evaluated on JD-FIT.
  • The noise-floor comparison is imperfect. Excl@5 for generative models is dominated by noise, so their bias cannot be bounded. "No significant effect" there reflects low power, which the authors acknowledge.
  • The comparison of Excl@5 against run-to-run Disag@5 is only approximately like-for-like.
  • The paper does not report the practical meaning of a 0.2–0.4 point score shift beyond rank displacement.
  • Three of the four models are from a single, recent, commercial ecosystem. JEV is a proprietary black box with no training-data or architecture transparency, which limits explanation of the observed biases.
  • The 1.13.0 version pinning is sensible but means the results age quickly.
  • The generative models receive no logprobs and output a single level, so their coarser scores create more ties. This is a structural handicap in rank-based metrics, which the paper partly discusses.

    • Potential Impact

    Practical. The work is directly relevant to compliance regimes (EU AI Act high-risk classification, NYC Local Law 144). It provides a replicable audit template and releases counterfactual CV datasets and evaluation code. Practitioners deploying cheap classifier-style models for screening get usable evidence on cost and stability together with caveats on bias.

    Methodological. The "noise-floor first" principle is a transferable and arguably underused norm in LLM fairness audits. Many existing hiring-bias papers use a single run.

    Scope. The impact is bounded by the novelty of the model class. The paper's relevance depends on whether "typed decision models" such as JEV become a durable category. The paper reads partly as an evaluation of one vendor's product, though Laya adds a second data point. The conclusion that JEV is "supported for large-scale screening subject to audits" could be read as endorsement of a commercial product by industry-affiliated authors.

    Timeliness & Relevance

    The topic is timely. AI Act high-risk obligations are approaching, interest in cost-efficient alternatives to LLMs is high, and fairness evaluation of non-generative decision models is sparse. The finding that determinism does not imply validity or fairness (Laya) is a useful cautionary result. The finding that post-2024 models tilt toward favoring historically disadvantaged groups is consistent with recent literature, so it adds corroboration rather than new surprise.

    Strengths

  • It frames bias against measurement instability, which improves the credibility of audits.
  • It combines fairness, validity, and cost, and so avoids evaluating fairness in isolation.
  • Appendices are detailed (full prompts, rubric, name-generation prompts, tests), and code and data are released.
  • It is honest about limitations, explicitly stating that the results do not certify any model as fair.
  • The Laya finding is an instructive negative result.

    • Limitations

  • Narrow bias data (one dataset, few names, binary gender, no intersectionality).
  • Lack of mechanistic insight into why names drive effects.
  • Possible confounds from name-generation choices.
  • Validity gains are modest. AUC is about 0.63–0.65 on JD-FIT, so all models are only moderately good screeners. The "comparable" claim rests on a low bar.
  • The significance of effects for JEV partly reflects its low noise, which makes small effects detectable. The paper notes this itself.
  • Bias mitigation is not explored.
  • Findings are tied to a particular rubric-based scoring pipeline, which may not match how these models are used in practice.

    • Overall

    This is a careful, practically useful audit with a transferable methodological message (calibrate bias against run-to-run variance). It is incremental in technique, with a modest empirical scope and a short expected shelf-life tied to specific model versions. I expect moderate citations in the fairness-in-hiring and LLM-auditing literature, and in industry audit practice.

    Rating:5.5/ 10
    Significance 5Rigor 6.5Novelty 5Clarity 7.5

    Generated Nov 4, 2026

    Comparison History (0)

    No comparisons yet.