What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Models
Payal Chandak, Victoria Alkin, David Wu, Maya Dagan, Taposh Dutta Roy, Maria Clara Saad Menezes, Ayush Noori, Nirali Somia
Abstract
Medicine is inherently pluralistic. Principles such as autonomy, beneficence, nonmaleficence, and justice routinely conflict, and such ethical dilemmas often sharply divide reasonable physicians. Good clinical practice navigates these tensions in concert with each patient's values rather than imposing a single ethical stance. The ethical values that large language models bring to medical advice, however, have not been systematically examined. We present a framework for auditing value pluralism in medical AI, comprising a benchmark of clinician-verified dilemmas and an attribution method that recovers value priorities directly from decisions. The ecosystem of frontier models spans physician-level value heterogeneity, and models discuss competing values in their reasoning (Overton pluralism) before committing to a decision. However, individual model decisions are near-deterministic across repeated sampling and semantic variations, failing to reproduce the distributional pluralism of the physician panel. Across benchmark cases, these consistent decisions reflect committed, systematic value preferences. While most model priorities fall within the natural range of inter-physician variation, some significantly underweight patient autonomy. A single LLM deployed without regard for its value priorities could amplify those priorities at scale to every patient it serves. Without explicit efforts to balance ethical perspectives with one or multiple models, these tools risk replacing clinical pluralism with a deployment monoculture.
AI Impact Assessments
(1 model)Scientific Impact Assessment
Core Contribution
This paper introduces a framework for auditing value pluralism in medical AI, consisting of two main components: (1) a benchmark of 50 clinician-verified clinical ethical dilemmas with structured value annotations grounded in principlism (autonomy, beneficence, nonmaleficence, justice), and (2) a revealed-preference attribution method that infers value priority distributions from binary clinical decisions rather than stated preferences. The central insight is that while individual LLMs discuss competing ethical values in their reasoning (Overton pluralism), they are near-deterministic decision-makers whose consistent choices reflect systematic value commitments — and some models significantly underweight patient autonomy relative to physician consensus. The concept of "deployment monoculture" — where a single LLM's fixed ethical stance replaces the natural diversity patients encounter across different physicians — is both novel and practically important.
Methodological Rigor
The methodology is generally sound and thoughtfully designed. Several elements stand out:
Benchmark construction follows a rigorous multi-stage pipeline with structural constraints (C1–C4) that guarantee genuine ethical trade-offs. The blinded two-stage physician review process and the formalization of value-difference vectors (Δᵢ) are well-motivated. The benchmark satisfies all three pluralistic benchmark desiderata of Sorensen et al.
Value attribution via logistic regression on value-difference vectors is elegant in its simplicity. The softmax temperature calibration using 500 synthetic Dirichlet agents (mean reconstruction JSD = 0.0086) provides convincing validation. The likelihood ratio test for non-uniform priorities is appropriate.
Decision consistency analysis is thorough, including 10 repeated queries per case at temperature 1.0, Spearman correlations against physician disagreement, and graded paraphrase experiments showing a dose-response pattern (negligible flip rate for surface rewording, sharp increase for value reversal). This effectively rules out the hypothesis that consistency is a phrasing artifact.
Calibration analysis uses bootstrap leave-one-out physician consensus with 10,000 iterations, producing a well-grounded reference distribution for interpreting model divergence.
However, several limitations warrant attention. The physician panel (N=20) is small and drawn from academic North American centers — a convenience sample that limits generalizability. The forced binary choice design, while enabling clean measurement, compresses nuanced clinical reasoning. The linear separability assumption in the logistic regression may miss non-linear value interactions. The benchmark's 50 cases, while diverse across clinical domains, provide limited statistical power for individual-level attribution (10 of 20 physicians fail to reject uniform weighting, which may reflect sample size rather than genuinely uniform priorities).
Potential Impact
Clinical AI deployment: The finding that some frontier models (GPT 5.2, Grok 4, Sonar Pro) significantly underweight autonomy — allocating 6–13% versus the physician consensus's 44% — has direct implications for patient-facing AI systems. This provides concrete, actionable evidence for regulators and healthcare systems considering LLM deployment.
AI alignment research: The paper operationalizes pluralistic alignment in a domain where ground truth is genuinely contested, contributing methodology that could extend to legal, policy, and other value-laden domains. The distinction between Overton, distributional, and steerable pluralism applied to a real-world domain is valuable.
Regulatory and economic implications: The connection drawn between physician values driving regional spending variation and the potential for algorithmic value encoding to similarly shift healthcare economics is provocative and policy-relevant.
Benchmark contribution: The benchmark itself, if released as planned, fills a gap in medical AI evaluation by moving beyond accuracy and safety to ethical alignment.
Timeliness & Relevance
This paper is exceptionally timely. LLMs are already being consulted by patients for medical advice, often without physician mediation. Existing medical AI benchmarks focus overwhelmingly on diagnostic accuracy and safety, leaving ethical alignment as a significant blind spot. The paper arrives at a moment when regulatory frameworks for medical AI are being actively debated, and the deployment monoculture concern resonates with broader discussions about foundation model homogenization.
Strengths
1. Conceptual clarity: The paper precisely defines multiple forms of pluralism and systematically tests each, yielding a nuanced picture (models achieve Overton but not distributional pluralism).
2. Revealed-preference approach: Inferring values from decisions rather than self-reports addresses the well-documented value-action gap in both humans and LLMs.
3. Calibration against human baseline: Rather than treating physician consensus as ground truth, the paper treats inter-physician variation as the reference distribution, appropriately reflecting genuine ethical pluralism.
4. Comprehensive model coverage: Testing 12 frontier models from diverse providers (OpenAI, Google, Anthropic, DeepSeek, Meta, etc.) gives ecological validity to ecosystem-level claims.
5. The deployment monoculture framing: This reframes the problem from "are models ethical?" to "does deployment preserve the value diversity patients need?" — a more sophisticated and actionable question.
Limitations
1. Western-centric bioethics: The benchmark is grounded in English-language principlism, which is not universally adopted across medical traditions. Cross-cultural validity is acknowledged but unexplored.
2. Temporal snapshot: Model versions change rapidly; value profiles may shift with updates, limiting the shelf life of specific findings.
3. No steerability experiments: The paper identifies steerable pluralism as "the most important" mode but defers testing it entirely to future work.
4. Ecological validity: Real clinical decisions involve iterative dialogue, patient-specific context, and multi-stakeholder input — the forced binary paradigm captures only a slice of ethical reasoning.
5. Attribution method limitations: The method recovers aggregate priorities across the benchmark but cannot assess whether models appropriately modulate values case-by-case (e.g., weighting autonomy more when patient capacity is high).
6. No causal analysis: The paper acknowledges that whether models have internal representations of principlist values that causally drive decisions remains open. The attribution is correlational.
Overall Assessment
This is a well-executed study that makes a meaningful contribution at the intersection of AI alignment, medical ethics, and responsible AI deployment. The framework is reproducible, the findings are specific and actionable, and the deployment monoculture concept offers a useful lens for policy discussions. The work would benefit from larger physician panels, cross-cultural replication, and steerability experiments, but as a foundational contribution it establishes important methodology and raises the right questions at the right time.
Generated May 19, 2026
Comparison History (26)
Paper 2 addresses a critical and timely issue at the intersection of AI ethics and healthcare—the systematic auditing of ethical value pluralism in medical LLMs. Its framework for detecting value biases in clinical AI has broad implications for AI safety, regulation, and deployment policy across healthcare systems worldwide. While Paper 1 presents solid engineering advances in biomedical agent planning, Paper 2's findings about deployment monoculture risks and autonomy underweighting are likely to influence policy discussions, clinical AI governance, and responsible AI research more broadly, giving it higher cross-disciplinary impact.
Paper 1 addresses a critical, high-stakes issue—ethical pluralism in clinical AI—with direct implications for patient care and medical AI safety. Its introduction of a clinician-verified benchmark and attribution method offers immediate real-world utility in preventing dangerous 'deployment monocultures' in healthcare. While Paper 2 presents a strong foundational framework for AI agent memory systems, Paper 1's interdisciplinary breadth (spanning machine learning, medical ethics, and healthcare policy) and its urgent societal relevance give it a higher potential for broad and profound scientific impact.
Paper 2 addresses a critical and timely concern—ethical value alignment of LLMs in healthcare—with broad interdisciplinary impact spanning AI safety, medical ethics, and policy. Its framework for auditing value pluralism is novel and applicable across domains. The finding that models exhibit near-deterministic ethical stances risks 'deployment monoculture' has immediate policy implications as medical AI scales. Paper 1, while technically solid, addresses a narrower engineering problem (skill library management) relevant primarily to the LLM agent research community, limiting its breadth of impact.
Paper 2 offers a foundational technical advancement in understanding and optimizing the internal reasoning mechanisms of large models. By addressing the instability of reinforcement learning without costly external verifiers, its methodology has broad, cross-disciplinary implications for the future development of all AI systems. While Paper 1 provides a crucial audit of AI in medical ethics, Paper 2's fundamental breakthrough in AI optimization is likely to drive widespread structural improvements across the entire field of artificial intelligence.
Paper 1 addresses the critical and timely issue of ethical value alignment in medical AI, a topic with broad societal implications as LLMs are increasingly deployed in healthcare. It introduces a novel auditing framework for value pluralism, bridging AI safety, medical ethics, and clinical practice. Its findings—that individual LLMs exhibit near-deterministic ethical stances potentially underweighting patient autonomy—have direct policy and deployment implications at scale. Paper 2, while technically sound, offers an incremental optimization improvement in a narrower subfield with limited broader impact beyond sparse optimization and adversarial attacks.
Paper 2 has higher potential impact: it introduces a novel, generalizable auditing framework for value pluralism in clinical LLMs with clinician-verified dilemmas and an attribution method, addressing a timely, high-stakes deployment risk (ethical monoculture) with clear real-world implications for medical AI governance and regulation. Its findings are broadly relevant across AI alignment, healthcare, ethics, and policy. Paper 1 is useful engineering work and comparative analysis of agent paradigms, but appears more incremental, tied to a specific framework and limited case-study evidence, with narrower cross-field impact.
Paper 1 likely has higher scientific impact: it introduces a clinician-verified benchmark plus an attribution method to audit value pluralism in medical AI, addressing a timely, high-stakes deployment risk (ethical monoculture) with broad relevance across medicine, AI safety, and policy. Its findings (near-deterministic model choices, underweighting autonomy) have direct implications for real-world clinical use and governance. Paper 2 is technically strong and useful for cheminformatics, but its impact is more domain-specific and may be superseded as multimodal LLM vision/chemistry tooling rapidly evolves.
Paper 2 likely has higher impact due to strong real-world stakes and timeliness: it audits ethical value pluralism in medical LLM advice, introduces a clinician-verified benchmark plus an attribution method, and yields actionable findings (near-deterministic model values, potential autonomy underweighting, deployment monoculture risk). This directly affects clinical governance, regulation, and deployment across healthcare systems, with broad relevance to AI ethics, evaluation, and human-AI decision-making. Paper 1 is novel and useful for optimization/AutoML, but its applications are more specialized and its societal leverage is typically lower.
Paper 2 addresses a critical and immediate challenge in AI deployment: clinical ethics and value pluralism. Its impact extends beyond technical AI communities into healthcare policy, medical ethics, and AI safety. While Paper 1 presents a highly innovative technical approach for EEG understanding, Paper 2's focus on auditing and preventing ethical monoculture in AI doctors has broader, more urgent real-world implications for global healthcare systems.
Paper 2 addresses a critical and timely issue—ethical value alignment in medical AI—with broader implications for AI safety, healthcare policy, and responsible deployment of LLMs. Its novel framework for auditing value pluralism is methodologically rigorous and applicable across many domains. Paper 1, while technically sound, shows modest improvements (cosine similarity of 0.181 vs 0.139) on a narrow EEG-to-text task and represents incremental progress in a niche area. Paper 2's findings about deployment monoculture risk have immediate policy relevance as medical AI scales rapidly.