Perception or Prejudice: Can MLLMs Go Beyond First Impressions of Personality?

Paper 1 is likely to have higher scientific impact due to a clearer methodological contribution and broader immediate relevance: it introduces a well-scoped new task (Grounded Personality Reasoning), a sizable multimodal benchmark with human verification, and diagnostic metrics that expose a field-wide failure mode (the “prejudice gap”) across 27 MLLMs. This offers a reusable evaluation framework for alignment/interpretability in social perception, affecting multimodal reasoning research and human-facing AI. Paper 2 is promising for applications, but “end-to-end researcher” systems often face reproducibility and evaluation ambiguity, making impact less certain.

Paper 1 introduces a more novel conceptual framework (Grounded Personality Reasoning) that exposes a fundamental limitation in MLLMs—the 'Prejudice Gap' between correct predictions and genuine understanding. This finding has broad implications across AI safety, fairness, and social cognition research. The three-tier evaluation with failure-mode metrics offers methodological innovation applicable beyond personality assessment. Paper 2, while rigorous and practically relevant, is more incremental in the negotiation benchmark space. Paper 1's discovery that 51% of correct ratings lack grounding is a striking finding likely to influence how the community evaluates and develops socially-aware AI systems.

Paper 1 addresses a fundamental, highly debated issue in AI—evaluating true reasoning versus pattern matching in LLMs. Its method of using formally verified, calibrated probes offers a rigorous, contamination-free framework applicable across multiple domains (math, physics, chemistry). This methodological innovation will likely influence a broad spectrum of LLM research. Paper 2, while important for human-AI interaction, focuses on a narrower niche (multimodal personality perception), making its overall scientific impact potentially more limited compared to general reasoning evaluation.

Paper 1 likely has higher impact due to stronger real-world stakes and immediate applicability: a large, expert-calibrated multinational benchmark directly targeting clinical reasoning and safety in dentistry, with quantified harm-risk analysis. Its scale (8,978 items; 88 countries; 14 specialties) and expert agreement suggest high rigor and strong potential to shape evaluation standards for medical LLM deployment and regulation. Paper 2 is novel and broadly relevant to multimodal social cognition, but its downstream applications are less urgent and safety-critical than clinical decision support.

Paper 1 has higher impact potential due to stronger novelty and methodological contribution: it introduces a new task (Grounded Personality Reasoning), a sizable multimodal dataset with timestamped evidence, and a multi-tier evaluation with concrete failure-mode metrics that can become standard for auditing “right answer for the right reason.” Its breadth spans multimodal grounding, social cognition, bias/prejudice analysis, and benchmark design, with clear real-world relevance for human-facing agents. Paper 2 is timely and psychometrically motivated, but is smaller in scale and less methodologically actionable beyond a fixed-item test.

Paper 1 addresses fundamental questions about reasoning, bias, and social cognition in Multimodal Large Language Models. By exposing the 'Prejudice Gap' and introducing a comprehensive dataset and metric suite for grounded reasoning, it has broad implications for AI safety, human-AI interaction, and model evaluation. Paper 2, while highly practical and methodologically rigorous for game development and multi-agent systems, focuses on a narrower domain (NPC control), making Paper 1's potential impact more profound across the wider AI research community.

Paper 2 introduces a novel benchmark and evaluation framework (MM-OCEAN) that addresses a fundamental gap in MLLM evaluation—whether models truly reason about personality or rely on superficial cues. Its contributions (new task formalization, dataset, and failure-mode metrics) are broadly applicable across the MLLM community and expose a striking 'Prejudice Gap' with implications for AI safety and trustworthiness. Paper 1, while technically sound, addresses a narrower problem (rare-token identity transfer in self-distillation) with more limited cross-field impact. Paper 2's timeliness, given widespread MLLM deployment, amplifies its potential influence.

Paper 1 likely has higher scientific impact because it contributes a concrete new task (GPR), a sizable multimodal dataset (MM-OCEAN) with human verification, and a multi-level evaluation framework with actionable failure-mode metrics, enabling immediate benchmarking and method development across many MLLMs. Its findings (e.g., “prejudice gap”) are timely for responsible deployment and could influence both model training and evaluation standards. Paper 2 is a valuable survey with a useful taxonomy and evaluation dimensions, but it is less methodologically empirical and primarily consolidates existing work, typically yielding lower direct research acceleration than a new benchmark/dataset.

Paper 2 identifies a critical, counter-intuitive inverse scaling phenomenon in high-stakes forecasting domains like epidemiology and finance. Discovering that increased model capabilities degrade performance on tail-risk predictions addresses urgent AI safety and reliability concerns. Its profound implications for real-world decision-making and LLM evaluation methodology give it a broader, more significant potential scientific impact than Paper 1's narrower focus on personality perception.

Paper 2 introduces a novel conceptual framework (Grounded Personality Reasoning) that addresses a fundamental question about whether MLLMs truly understand or merely pattern-match—a question with broad implications across AI safety, fairness, and deployment. Its contributions (new task formalization, large dataset, comprehensive evaluation with novel failure-mode metrics, and benchmarking 27 models) are methodologically rigorous and reveal a striking 'Prejudice Gap' finding. This has broader cross-field impact (social AI, psychology, fairness/bias) compared to Paper 1, which provides a valuable but more domain-specific benchmark for spreadsheet tasks in finance.