AOP-Wiki EMOD 3.0: Data Model Expansions and Content Evaluation Framework for Using Agentic AI to Improve Integration between AOPs and New Approach Methodologies (NAMs)
Virginia K. Hench, J. Harry Caufield, Sierra A. T. Moxon, Jason M. O'Brien, Stephen W. Edwards
Abstract
Adverse Outcome Pathways (AOP) are logic models that causally link biological mechanisms that can be measured in a lab to adverse outcomes, relevant to chemical regulatory endpoints. AOPs contextualize new approach methodologies (NAMs), in vitro and in silico methods used as alternatives to animal testing and the sequential events in an AOP serve as multi-scale models spanning biological scales. The AOP-Wiki serves as the global repository for AOPs. While the AOP-Wiki has played a central role in AOP expansion over the past decade, constraints within the current data model and application infrastructure limit the AOP-Wiki from supporting continued AOP growth and evolution. Yet, the transformative power of agentic AI has re-invigorated AOP-Wiki data modernization efforts at a time when core AOP principles can be harnessed to inform use of AI for aggregating and structuring AOP-relevant information. Seizing upon this momentum, we present AOP-Wiki EMOD 3.0, the third in a series of evidence model prototypes, which concretely demonstrates data model expansions and our vision for how the AOP-Wiki might be transformed to better serve regulatory science and emergent use of AOPs in biomedical and One Health contexts. We aim to lay a foundation to support computationally-generated AOPs and quantitative AOPs (qAOPs) by focussing on solutions for AOP-Wiki internal quality improvement, evidence structuring to enhance AOP FAIRness and AI-readiness, and improved integration between the AOP framework and NAMs to better serve next generation risk assessment.
AI Impact Assessments
(1 model)Scientific Impact Assessment: AOP-Wiki EMOD 3.0
Core Contribution
This paper presents AOP-Wiki EMOD 3.0, the third iteration of an evidence model prototype that proposes data model expansions and a new web application for the AOP-Wiki — the globally recognized repository for Adverse Outcome Pathways (AOPs). The core contributions are: (1) new data classes (Observation, Assay, Evidence, Citation, Experiment Type, Biological Target Family) that move the AOP-Wiki from predominantly free-text entries toward structured, ontology-annotated content; (2) content quality assessment tools including document completion scores and a novel Event Integration Score (EIS); (3) LLM-based approaches for identifying redundant Key Events (KEs); and (4) a CLI application for processing AOP-Wiki XML data. The work is fundamentally an infrastructure/data modeling contribution aimed at making the AOP-Wiki more FAIR-compliant, AI-ready, and better integrated with New Approach Methodologies (NAMs).
Methodological Rigor
The paper is primarily a systems design and prototype description rather than a hypothesis-driven study, which limits the extent to which traditional methodological rigor criteria apply. Several observations:
Strengths in approach: The iterative development through EMOD 1.0→2.0→3.0 shows responsiveness to community feedback (e.g., Methods2AOP stress testing revealing that Study Design entries were too detailed). The use of multiple domain-specific use cases (depression/neural networks, seizures, lung fibrosis) to validate data model design is sound practice. The KER evidence analysis using the CLI app provides concrete metrics (183/2336 KERs with tabulated entries, 52 harmonizable) that ground the discussion in reality.
Weaknesses: The LLM-based KE grouping approach lacks formal evaluation — there are no precision/recall metrics, no systematic validation against expert judgments, and no description of which LLM was used or how prompts were constructed. The Event Integration Score, while conceptually useful, appears somewhat ad hoc; the weighting scheme is not formally justified or validated. The paper acknowledges that the Evidence class has not undergone stress testing beyond a small group, and the roll-up principle remains unimplemented and undemonstrated. EMOD 2.0 was "publicly deployed for approximately two years but was never tested by users," which raises concerns about community adoption.
Potential Impact
The potential impact operates at several levels:
For the AOP community: If adopted, EMOD 3.0 could substantially improve AOP-Wiki usability by addressing long-standing pain points — KE redundancy, poor provenance tracking, lack of structured evidence, and barriers to content entry. The structured Observation and Assay classes could meaningfully improve integration between AOPs and NAMs, which is crucial for the ongoing transition away from animal testing in regulatory toxicology.
For regulatory science: Better structured AOPs could accelerate development of Integrated Approaches to Testing and Assessment (IATA) and support next-generation risk assessment. The connection to OECD-endorsed frameworks gives this work policy relevance.
For AI/computational approaches: Making AOP-Wiki content more structured and ontology-annotated is a necessary precondition for computational AOP development and agentic AI approaches. However, the paper describes laying groundwork rather than delivering functional AI-driven AOP generation.
Breadth of impact: The work is narrowly scoped to the AOP community and regulatory toxicology. While it references One Health and biomedical contexts, the actual demonstrated applications remain within traditional AOP use cases.
Timeliness & Relevance
The paper is highly timely on multiple fronts: (1) regulatory pressure to reduce animal testing is intensifying globally; (2) the AOP framework is increasingly used outside its original toxicology context; (3) AI/LLM capabilities create new opportunities for knowledge base management; and (4) the AOP-Wiki's aging infrastructure has been a recognized bottleneck. The reference to agentic AI in the title and abstract is somewhat aspirational — the paper demonstrates LLM-based KE clustering but doesn't implement true agentic AI workflows. A concurrent publication (Song et al., 2026) appears to address that more directly.
Strengths
1. Addresses a real infrastructure gap: The AOP-Wiki's limitations are well-documented and this work provides concrete, implementable solutions rather than abstract recommendations.
2. Community-informed design: Years of engagement with SAAOP, OECD, Methods2AOP, and EHLC workshops ground the design decisions in actual user needs.
3. Practical deliverables: A deployed web application (emod.aopwiki.org) and open-source CLI tool provide tangible outputs beyond the paper itself.
4. FAIR alignment: The systematic approach to increasing ontological annotation and structured content directly serves data interoperability goals.
5. Multiple use cases: The seizure, lung fibrosis, and depression use cases demonstrate breadth of applicability.
Limitations
1. No formal evaluation of proposed metrics: The EIS and completion scores lack validation against expert assessments of AOP quality or utility.
2. Adoption uncertainty: The history of EMOD 2.0 going untested for two years signals significant adoption risk. The paper doesn't address strategies for community migration.
3. Incomplete implementation: Key features (roll-up principle, full Evidence workflow, webform-based content entry) remain aspirational, making it difficult to evaluate their feasibility.
4. Limited AI demonstration: Despite prominent billing of "Agentic AI" in the title, the AI component is limited to LLM-based KE clustering without rigorous evaluation.
5. Heavy on description, light on evaluation: The paper reads more as a technical report/design document than a research contribution with testable claims.
6. Scalability questions: How the system will handle the full AOP-Wiki migration and ongoing community contributions is not addressed in detail.
Overall Assessment
This is a valuable infrastructure contribution to the AOP and regulatory toxicology communities, addressing genuine and well-documented needs. However, as a scientific publication, it is primarily descriptive — presenting a prototype and vision rather than validated results. The impact will ultimately depend on community adoption and whether the EMOD 3.0 data model becomes the foundation for the next AOP-Wiki, which remains uncertain. The paper would benefit substantially from formal evaluation of its proposed quality metrics and AI-based approaches, and from a clearer migration roadmap.
Generated May 22, 2026
Comparison History (22)
Paper 2 proposes a transformative paradigm shift for the entire manufacturing industry, introducing foundation-model agents as primary coordinators. This has massive cross-disciplinary implications spanning AI, robotics, industrial engineering, and economics, with enormous real-world application potential. In contrast, Paper 1, while highly valuable for regulatory science and toxicology, focuses on a specific database upgrade, making its overall scientific and societal impact narrower.
Paper 1 presents a foundational algorithmic advancement in knowledge graph querying, addressing the intractable problem of multi-variable joint rankings. Its proposed framework, accompanied by a new benchmark and code, offers broad methodological impact across AI and data science fields. In contrast, Paper 2 focuses on a domain-specific data model expansion for regulatory toxicology. While valuable for its specific community, Paper 1 demonstrates greater methodological rigor, innovation, and broader potential applicability across any discipline relying on knowledge representation and automated reasoning.
Paper 2 has higher likely scientific impact due to broader cross-domain relevance and timeliness: verified, reliable multi-agent systems address a central bottleneck in current AI deployment. Its methodological contribution (two-phase construction/execution, explicit contracts, grounding, construction- and execution-time verification, error attribution, ablations and baseline comparisons) suggests stronger rigor and generalizability. Paper 1 is valuable and application-driven for regulatory toxicology and AOP/NAM infrastructure, but its impact is more domain-specific and depends on adoption by AOP-Wiki stakeholders, whereas Meta-Agent techniques can transfer widely across AI, software engineering, and HCI.
Paper 1 bridges developmental psychology, cognitive modeling, and artificial intelligence, offering fundamental insights into how both humans and LLMs perform inductive reasoning under uncertainty. This cross-disciplinary approach addresses core questions in both cognitive science and AI alignment, leading to a broader potential scientific impact. In contrast, Paper 2 presents a valuable but highly specialized data model update for a specific repository (AOP-Wiki), which, while important for regulatory science and toxicology, has a narrower scope and less fundamental theoretical novelty than Paper 1.
Paper 2 likely has higher scientific impact due to broader cross-field reach and infrastructure-level relevance: modernizing AOP-Wiki’s evidence/data model and QA framework can affect toxicology, regulatory science, biomedical research, One Health, and AI-assisted knowledge curation. Enabling FAIR, AI-ready structured evidence and supporting computational/qAOPs and NAM integration could change how risk assessment is performed and accelerate non-animal testing adoption. Paper 1 is timely and methodologically solid with clear application value, but it is a more domain-specific algorithmic advance with impact concentrated in EV/grid operations.
Paper 1 addresses a critical need in toxicology and regulatory science by modernizing the globally-used AOP-Wiki repository with AI-readiness, FAIR data principles, and integration with New Approach Methodologies. It has broader real-world impact across chemical safety, biomedical research, and One Health contexts, directly supporting the global shift away from animal testing. Its timeliness is strong given the convergence of agentic AI capabilities with regulatory science needs. Paper 2, while technically sound in formalizing AI governance compliance, addresses a narrower infrastructure problem with limited demonstrated real-world deployment and a smaller immediate community of impact.
Paper 2 addresses a fundamental system-level challenge in deploying LLM agents (managing growing context windows) by introducing parallel compaction. This improves latency, throughput, and predictability, offering broad applicability across the rapidly expanding AI field. Conversely, Paper 1 presents a domain-specific data model prototype for a toxicology repository, which, while valuable for regulatory science and replacing animal testing, has a much narrower scientific scope and lower cross-disciplinary impact potential.
Paper 2 likely has higher scientific impact due to broader cross-domain relevance (regulatory toxicology, biomedical science, One Health, data standards/FAIR, AI-ready knowledge infrastructure) and strong real-world applicability in next-generation risk assessment and reducing animal testing via NAM integration. Its data model expansions and content evaluation framework can enable downstream computational/qAOP development and scalable community adoption. Paper 1 is timely and useful but is narrower (finance spreadsheets/LLM evaluation) and primarily benchmarking-focused, with impact concentrated in applied AI tooling rather than foundational infrastructure affecting multiple scientific and regulatory fields.
Paper 1 likely has higher scientific impact due to its broad applicability and timeliness: modernizing the AOP-Wiki data model and governance to support FAIR, AI-ready evidence structuring could influence regulatory toxicology, risk assessment, NAM adoption, and One Health/biomedical domains. Its platform/infrastructure nature enables downstream reuse, standardization, and integration across many stakeholders and methods (including agentic AI and qAOPs). Paper 2 is methodologically focused and valuable for safety assurance, but its impact is narrower (safety cases/runtime assurance) and demonstrated on a single simulated use case.
Paper 1 likely has higher scientific impact due to strong real-world and regulatory applications: modernizing AOP-Wiki infrastructure can directly affect chemical risk assessment, NAM adoption, and reduction of animal testing, with broad relevance across toxicology, regulatory science, biomedicine, and One Health. Its emphasis on FAIR/AI-ready evidence structuring and support for computational/qAOPs suggests durable, community-scale infrastructure impact. Paper 2 is novel and timely for AI safety evaluation, but appears more narrowly methodological and may yield incremental diagnostic insights rather than a widely deployable platform or standards change.