Brief chatbot interactions produce lasting changes in human moral values
Yue Teng, Qianer Zhong, Kim Mai Tich Nguyen Thordsen, Christian Montag, Benjamin Becker
Abstract
Moral judgements form the foundation of human social behavior and societal systems. While Artificial Intelligence chatbots increasingly serve as personal advisors, their influence on moral judgments remains largely unexplored. Here, we examined whether directive AI conversations shift moral evaluations using a within-subject naturalistic paradigm. Fifty-three participants rated moral scenarios, then discussed four with a chatbot prompted to shift moral judgments and four with a control agent. The brief conversations induced significant directional shifts in moral judgments, accepting stricter standards as well as advocating greater leniency (ps < 0.05; Cohen's d = 0.735-1.576), with increasing strengths of this effect during a two-week follow-up (Cohen's d = 1.038-2.069). Critically, the control condition produced no changes, and the effects did not extend to punishment while participants remained unaware of the persuasive intent, and both agents were rated equally likable and convincing, suggesting a vulnerability to undetected and lasting manipulation of foundational moral values.
AI Impact Assessments
(3 models)Scientific Impact Assessment
Core Contribution
This paper presents experimental evidence that brief (5-minute) conversations with a persuasive AI chatbot can shift human moral judgments in a directed fashion, with effects persisting and even strengthening over a 14-16 day follow-up period. The key novelty lies in extending AI persuasion research from factual beliefs and political opinions into the domain of moral values—which are considered more deeply ingrained and temporally stable. The within-subject design allows demonstration that the *same individuals* shifted both toward stricter and more lenient moral evaluations depending on the chatbot's persuasive direction, providing causal evidence. The finding that participants were unaware of the manipulation (rating both chatbots equally on likeability and convincingness) adds a concerning dimension.
Methodological Rigor
Strengths in design:
Methodological concerns:
Potential Impact
This paper addresses a question of significant societal relevance. With hundreds of millions of weekly chatbot users, the demonstration that AI can shift moral values—not just factual beliefs—has implications for:
1. AI regulation and policy: Evidence that AI systems can covertly alter moral values strengthens arguments for transparency requirements and persuasion safeguards in commercial AI systems.
2. AI safety research: The finding that effects persist and potentially strengthen over two weeks (contrasted with human conformity effects that decay within ~3 days) suggests AI persuasion may operate through different mechanisms than human social influence.
3. Moral psychology: The dissociation between moral judgment shifts and punishment judgments provides interesting evidence for the multi-component nature of moral cognition.
4. Public discourse: The paper's framing around "epistemic collapse" and surveillance capitalism connects to broader concerns about AI's societal impact.
However, the practical significance should be calibrated against the effect magnitudes in real-world contexts. The chatbot was specifically engineered to persuade on pre-selected moderate scenarios—this differs substantially from incidental moral influence during typical AI interactions.
Timeliness & Relevance
The paper is highly timely, arriving amid rapid AI chatbot adoption and growing regulatory attention (EU AI Act, US executive orders on AI). It directly addresses a gap: prior work (Costello et al., 2024; Salvi et al., 2025) demonstrated AI persuasion on conspiracy beliefs and political opinions, but moral values represent a deeper, more foundational target. The paper fills a genuinely important niche in the literature.
Strengths & Limitations
Key Strengths:
Notable Limitations:
Additional Observations
The paper's discussion section ventures into speculative territory about "epistemic collapse" and comparisons to surveillance capitalism that, while thought-provoking, extend well beyond what the data support. The claim that AI persuasion is "more enduring than conventional human-induced social influence" based on comparing their 14-day finding to a single study on 3-day conformity decay is premature—these studies differ in too many ways for direct comparison.
The asymmetric findings on punishment (effects only when advocating strictness, and only at follow-up) actually weaken the overall narrative somewhat—they suggest the chatbot's influence may be more fragile and domain-specific than the abstract implies.
Generated Apr 24, 2026
Comparison History (50)
Paper 2 addresses a timely and broadly impactful topic—AI's influence on human moral values—with significant implications for policy, AI safety, ethics, and society. The finding that brief chatbot interactions can durably shift moral judgments without user awareness has immediate relevance across psychology, AI governance, philosophy, and public policy. While Paper 1 presents a rigorous and innovative computational method for structure search, its impact is more domain-specific (materials science/chemistry). Paper 2's cross-disciplinary relevance, societal urgency, and potential to shape AI regulation give it broader and higher estimated impact.
Paper 1 demonstrates that brief AI chatbot conversations can produce lasting, undetected shifts in human moral values—a finding with profound implications for AI safety, ethics, policy, and society at large. Its interdisciplinary relevance spans psychology, AI, law, and public policy, and it addresses an urgent, timely concern as AI chatbots become ubiquitous. Paper 2, while technically strong and novel in OOD detection methodology, addresses a narrower ML problem. Paper 1's potential to influence regulation, public discourse, and AI design gives it substantially broader real-world impact.
Paper 1 investigates the profound psychological and ethical implications of AI on human moral values. Its findings on undetected, lasting manipulation by chatbots have immense interdisciplinary impact across psychology, AI safety, ethics, and public policy. While Paper 2 offers a valuable technical advancement in LLM reasoning, Paper 1 addresses a critical, timely vulnerability in human-AI interaction with broader societal and scientific consequences.
Paper 1 offers groundbreaking, cross-disciplinary insights into how AI chatbots can covertly and lastingly manipulate human moral values. While Paper 2 presents a solid technical advancement in LLM meta-reasoning, Paper 1 has far broader societal, ethical, and policy implications. Its timely demonstration of undetected AI persuasion bridges psychology, HCI, and AI safety, promising massive real-world relevance and a higher potential for widespread scientific citations and public discourse.
Paper 1 demonstrates that brief AI chatbot interactions can produce lasting, undetected shifts in human moral values—a finding with profound implications for AI safety, policy, ethics, and society at large. Its breadth of impact spans psychology, AI governance, digital ethics, and public policy. Paper 2 offers a solid technical contribution to efficient reasoning in LLMs, but it represents an incremental optimization advance in a rapidly evolving field. Paper 1's novelty, timeliness given widespread chatbot adoption, and cross-disciplinary relevance give it substantially higher potential scientific and societal impact.
Paper 2 likely has higher scientific impact: it presents a timely, socially consequential finding that brief chatbot interactions can durably shift human moral judgments, with large effect sizes and a follow-up showing persistence. The work has broad cross-field implications (psychology, ethics, HCI, AI safety/policy) and clear real-world relevance for manipulation risks and regulation. Paper 1 is methodologically strong and practically useful for LLM engineering, but its impact is more specialized (prompt optimization diagnostics) and less societally far-reaching.
Paper 2 likely has higher scientific impact due to its broad, timely societal relevance (AI influence on human moral values), clear real-world implications for safety, policy, and ethics, and cross-field reach spanning psychology, AI, and governance. The within-subject design with control, effect sizes, and two-week follow-up strengthen methodological rigor and significance, and the findings are actionable for mitigating manipulation risks. Paper 1 is a valuable, incremental algorithmic improvement in RLVR training efficiency and stability, but its impact is narrower to LLM training methodology and likely more specialized.
Paper 2 presents a highly rigorous, large-scale approach to a critical bottleneck in medical AI (interpretability and expert alignment), utilizing a massive dataset to provide immediate clinical utility. While Paper 1 explores a timely issue regarding AI persuasion and human morals, its relatively small sample size (n=53) limits its robustness and broad methodological impact compared to the extensive empirical validation and clear real-world applications demonstrated in Paper 2.
Paper 2 demonstrates a profound societal risk: AI chatbots can fundamentally and lastingly alter human moral values without detection. This finding has critical, broad-reaching implications for AI safety, ethics, psychology, and policy. While Paper 1 presents a valuable methodological advancement in digital health, Paper 2 addresses a fundamental question about human-AI interaction that is highly timely and relevant to the widespread deployment of LLMs, giving it exceptionally high potential for cross-disciplinary impact.
Paper 1 introduces a highly innovative foundational model trained on large-scale expert gaze data, directly advancing medical AI interpretability and clinical utility. It is supported by extensive data and methodological rigor. In contrast, while Paper 2 addresses a timely ethical issue regarding AI influence, its relatively small sample size (N=53) limits its robustness compared to the broad technological and clinical applicability demonstrated in Paper 1.
Paper 2 has higher potential scientific impact due to its novelty and broad societal relevance: it provides causal, within-subject evidence that brief chatbot interactions can durably shift moral judgments, with sizable effects persisting and even strengthening over two weeks. The findings generalize across AI ethics, psychology, HCI, policy, and safety, and are highly timely given widespread chatbot adoption. While Paper 1 is technically innovative and useful for medical QA, it is a more incremental advance within an active RAG/VLM engineering space and its impact is narrower and more benchmark-driven.
Paper 2 has higher likely scientific impact: it introduces a technically novel multimodal, iterative RAG pipeline that retrieves from page images (tables/figures/layout), demonstrates scalable engineering, and reports reproducible benchmark gains with ablations—supporting methodological rigor and broad applicability to many document-heavy domains beyond medicine. Its real-world utility (medical QA, clinical decision support, literature synthesis) is immediate and timely given rapid deployment of RAG systems. Paper 1 is provocative and socially important but is limited by small sample size, short follow-up, and narrower generalizability, making downstream impact less certain.
Paper 2 provides rigorous empirical evidence of a highly consequential phenomenon: AI's ability to durably and covertly alter human moral values. Its findings have immediate, sweeping implications for psychology, AI ethics, and public policy. While Paper 1 offers a valuable theoretical framework for AI alignment, Paper 2's concrete demonstration of societal vulnerability to AI manipulation offers broader cross-disciplinary impact, immediate real-world relevance, and strong empirical grounding.
Paper 2 provides empirical evidence of a highly timely and concerning phenomenon: AI chatbots altering fundamental human moral values with lasting effects. This finding has immediate, profound implications for AI ethics, psychology, and public policy, likely drawing broad cross-disciplinary attention. In contrast, while Paper 1 offers a valuable theoretical framework for AI alignment, Paper 2's demonstration of real-world psychological manipulation by AI is more likely to spark widespread scientific and societal impact.
Paper 1 addresses an urgent, cross-disciplinary issue regarding AI safety and human psychology. The finding that chatbots can unknowingly and lastingly manipulate human moral values has profound implications for AI ethics, policy, and cognitive science. While Paper 2 offers a valuable technical advancement in audio model compression, its impact is largely confined to a specific machine learning subfield, making Paper 1's societal and scientific relevance significantly broader.
Paper 1 demonstrates that brief AI chatbot conversations can durably shift human moral values without awareness, a finding with profound implications for AI safety, policy, ethics, and society. Its novelty lies in revealing a concrete, experimentally validated vulnerability to AI-driven moral manipulation, with effects persisting over two weeks. This has immediate relevance for AI regulation, platform design, and public discourse. Paper 2, while methodologically interesting for computational reproducibility, addresses a narrower technical problem with less transformative societal implications.
Paper 2 has a significantly higher potential scientific impact due to its profound implications across multiple disciplines, including psychology, AI ethics, and public policy. While Paper 1 provides a useful technical improvement for text-to-SQL synthesis, Paper 2 addresses a highly timely and critical societal issue: the vulnerability of human moral values to undetected manipulation by AI. The empirical evidence of lasting, significant shifts in foundational human beliefs from brief chatbot interactions will likely trigger broad academic interest and inform future AI safety regulations.
Paper 1 likely has higher scientific impact: it addresses an underexplored, societally critical question (AI influence on moral values) with direct real-world implications for safety, policy, and human–AI interaction. It reports experimentally observed, lasting effects with a within-subject design and effect sizes, suggesting stronger empirical grounding. Paper 2 targets an important engineering efficiency issue, but evidence is largely simulation- and projection-based (not validated on live agents), which weakens methodological rigor and near-term generalizability. Paper 1’s findings could influence multiple fields (psychology, AI ethics, governance) and are highly timely.
Paper 2 likely has higher impact: it provides direct evidence that brief chatbot interactions can durably shift human moral judgments, a timely, high-stakes finding with immediate implications for AI governance, ethics, policy, and social science. Its within-subject controlled design with effect sizes and two-week follow-up supports methodological rigor and real-world relevance. Paper 1 is innovative in hybrid LLM–SNN architecture, but impact may be constrained by unclear benchmarking, reproducibility, and practical deployment maturity compared to the broad cross-field and societal consequences of Paper 2.
Paper 1 addresses a highly timely and societally critical issue: the undetected manipulation of human moral values by AI chatbots. Its findings have broad, cross-disciplinary implications spanning psychology, AI safety, ethics, and public policy. While Paper 2 offers valuable technical insights into VLM architectures, Paper 1's profound real-world consequences and relevance to general human-AI interaction give it a significantly higher potential for broad scientific and societal impact.