Hyperbolic Inverse Reinforcement Learning

Iulia Rusu, Anoop Praturu, Sergei Gepshtein, Tatyana Sharpee

Oct 30, 2026arXiv:2610.42104v1

cs.LG

Scorecard· 16/16

5.0/10 impact

Neat, cleanly presented application of hyperbolic distance rewards to MaxEnt IRL with a good low-parameter result, but evidence is small-scale, baselines are limited, and the gradient is approximated.

Abstract

Inverse reinforcement learning (IRL) seeks to infer the reward function underlying observed behavior, but its predictive accuracy and interpretability depend on how that reward is computed from the underlying features. While prior IRL reward functions have been implemented through a variety of techniques including linear weights, neural networks, or other function approximators, none make assumptions about the geometry of the underlying feature space. Inspired by geometric organization observed in neural circuits, we introduce Hyperbolic inverse reinforcement learning (Hyperbolic IRL), which parameterizes reward as a function of Hyperbolic distance between the feature vector and a learned parameter vector, as well as its Euclidean distance variant. We tested our methods against a range of linear and nonlinear IRL baselines under both deterministic and stochastic policies. Our Hyperbolic approach achieved comparable performance to state-of-the-art models on established benchmarks and excelled in the low data limit and across dense environments. Applied to human navigation among multiple exhibits in a museum room, Hyperbolic IRL reveals reward structure that depends not only on individual exhibit-distance features, but also on non-separable conjunctions between them. Together, these results demonstrate that a Hyperbolic reward parameterization can improve both the predictive accuracy and interpretability of IRL, providing a framework for studying how non-separable, conjunctive structure can begin to explain behavior.

AI Impact Assessments

(1 model)

Core Contribution

The paper proposes Hyperbolic IRL, a MaxEnt IRL variant in which reward is the negative squared geodesic distance between a state's feature vector and a learned parameter vector. The distance is computed on a Lorentz-model hyperboloid with tunable curvature. A Euclidean-distance variant is also given. The idea transfers the "distance instead of dot product" approach of Praturu & Sharpee (2025) from matrix factorization to reward parameterization. Main claims:

Better performance in low-data and dense-object Objectworld settings.

Better trajectory prediction on museum visitor data, with only 15 parameters against 1,601 for DeepIRL.

A reward surface with "conjunctive" (non-separable) feature interactions that correlates with behavioral co-attendance.

The novelty is a new combination, hyperbolic geometry applied to IRL reward. It is a modest conceptual step, motivated more by neuroscience analogy than by a derived need.

Methodological Rigor

Strengths:

The Objectworld experiments vary data amount and object density, evaluate under both deterministic and stochastic policies, and include transfer to held-out environments, with 16 repeats.

The Mantel-style permutation test and the distance-controlled partial correlation are thoughtful for the co-attendance analysis. The authors openly report that naive p-values were inflated by about three orders of magnitude.

Weaknesses:

The hyperbolic gradient is not exact. The 1/sinh(d) factor is dropped "in practice" (Eq. 64), so the optimizer no longer follows the stated MaxEnt likelihood. This is a significant theoretical gap, and no ablation shows how it affects results.

Curvature is selected from a discrete set (−1, −4, and −25 in the appendix). The method is evaluated against the true Objectworld reward, which is a conjunction of two distances. A distance-to-parameter reward could plausibly match this structure by design, and the benchmark may favor the method.

Baselines are thin. They are classic MaxEnt IRL, DeepIRL, and a Euclidean variant. There are no GP-IRL, adversarial IRL, or other modern baselines, and no tuned small-MLP baseline. DeepIRL with 1,601 parameters trained on few demos is a weak comparator for a parameter-efficiency claim.

Euclidean squared distance reduces to a linear reward plus a quadratic penalty, so it is nearly a linear model. That explains why it behaves like the classic model.

The museum data are very small: 26 trajectories total, 4 test trajectories, and SEM computed over 4. There is no significance testing between models on ADE and FDE. Rollout-based metrics from five-action discrete policies are noisy.

The "conjunctive" interpretation is partly circular. Hyperbolic distance in the Lorentz model contains the term θ₀ϕ₀, with ϕ₀ = √(1+‖ϕ‖²). This couples all features by construction, so non-separability is a built-in property rather than a discovery. The co-attendance correlation (r ≈ 0.5, permutation p ≈ 0.003) is nevertheless interesting. However, it rests on 13 exhibits, and trajectory-level non-independence is acknowledged as unaddressed. The appendix says "ruling out" proximity, which is reasonable, but the mechanism is unclear. A stronger test would compare against flexible nonlinear baselines, since DeepIRL also gives a non-additive surface.

Ground-truth validation of interpretability exists only in synthetic data, and it is not examined there.

Potential Impact

The most likely users are computational neuroscientists and behavioral scientists who use IRL on small datasets. A parameter-efficient, non-additive reward model is useful to them.

In mainstream ML and RL, impact is likely limited. Performance is "comparable" on established benchmarks, and no standard IRL benchmarks beyond Objectworld (such as MuJoCo or imitation benchmarks) are used.

The lasting contribution is more likely the idea that reward geometry is a design choice than the specific model.

Reuse is plausible because the method is a drop-in reward head. The numerical-stability fixes in the appendix, such as capping the geodesic step, are practical details.

Timeliness & Relevance

Hyperbolic representation learning is established, and IRL for animal and human behavior is growing (Ashwood, Ke, and others). The combination is timely for neuro-AI crossover. In mainstream IRL the field has moved toward adversarial and large-scale preference learning, so this is a niche direction.

Strengths

A simple, unified framework in which curvature is a hyperparameter.

Large parameter savings.

A real human dataset, with an honest statistical correction.

The paper is clear and well organized, with transparent disclosure of approximations and limitations (BIC omitted, trajectory-level dependence).

Limitations

The approximate gradient, which departs from the exact MaxEnt gradient.

Small-scale evaluation and a benchmark that may favor the method.

Limited baselines.

The causal link between hyperbolic geometry and the performance gains is hypothesized (exponential volume packing, conjunctive coupling) but not tested. No ablation separates curvature from the extra θ₀ coordinate or from the nonlinearity of the arcosh² form.

The neuroscience motivation (hyperbolic neural circuits) is not tested on neural data.

The claim of "improved interpretability" is not backed by any human or ground-truth interpretability evaluation.

Reproducibility is moderate. The method is specified in detail and the museum data are public, but code is not mentioned, and hyperparameters and the Objectworld settings for the museum experiments are only partly given.

Overall

This is a solid but incremental, niche contribution. The idea is neat and the empirical analysis includes careful statistical touches. The evidence is limited by small data, a favorable synthetic benchmark, and an approximate optimizer. It will probably be cited within the neuro-IRL and geometric-ML communities but is unlikely to reshape IRL practice.

Rating:5/ 10

Significance 4.5Rigor 4.5Novelty 5.5Clarity 7

Generated Nov 3, 2026

Comparison History (0)

No comparisons yet.

Scorecard· 16/16

5.0/10 impact