Probabilistic Machine Learning for Short-Term Atmospheric Seeing Forecasting

Mary-Joe Medlej, Rahul Srinivasan, Simon Prunet, Aziz Ziad, Christophe Giordano

Oct 21, 2026arXiv:2610.29654v1

cs.LGastro-ph.IM

Scorecard· 16/16

3.5/10 impact

Sensible but thin application benchmark whose headline probabilistic claims lack quantitative probabilistic metrics, with limited novelty and a niche audience.

Abstract

Accurate short-term forecasting of atmospheric seeing is of significant interest for astronomical observations and free-space optical communication systems, where rapid variations in optical turbulence can affect system performance and operational planning. In this work, we benchmark machine learning models for predicting atmospheric seeing up to two hours in advance, with a particular focus on probabilistic forecasting and uncertainty quantification. We compare two probabilistic approaches, Gaussian Processes (GPs) and a normalizing-flow- based model (FloTS), against deterministic recurrent neural network (RNN) and long short-term memory (LSTM) baselines. All models are trained and evaluated on seeing measurements from the Maunakea summit. We assess the models using both deterministic metrics, including the root mean square error and the Pearson correlation coefficient, and probabilistic metrics that evaluate the quality and calibration of predictive uncertainty estimates. The results show that deep learning approaches outperform standard recurrent networks for short-term forecasting, with LSTM and FloTS achieving the best overall predictive accuracy. While Gaussian Processes provide a useful probabilistic baseline, their predictive distributions are limited by the Gaussian assumption. FloTS combines competitive forecasting accuracy with flexible uncertainty quantification and produces the most informative probabilistic forecasts.

AI Impact Assessments

(1 model)

Core Contribution

This is a short conference-style (SPIE-type) benchmarking paper. It compares four models for 10–120 minute atmospheric seeing forecasting on a 15-year Maunakea DIMM dataset (2009–2024): a deterministic RNN, a deterministic LSTM, a Gaussian Process with an exponential kernel, and FloTS, a conditional Masked Autoregressive Flow with an LSTM context encoder. The novelty is mainly the application of probabilistic forecasting (normalizing flows plus post-hoc calibration) to astronomical seeing, where earlier work (ARIMA, random forests, LSTMs, Cherubini/Lyman/Businger) focused on point forecasts. The paper reuses existing FloTS and a temperature-scaling calibration borrowed from the lead author's gravitational-wave work. It introduces no new algorithm and offers no new theoretical insight.

Methodological Rigor

The experimental design is simple and has several gaps:

The abstract promises "probabilistic metrics that evaluate quality and calibration," but the results section reports only RMSE and Pearson correlation, summarized in a min-max-normalized composite score. No CRPS, log-score, PIT histograms, coverage tables, or sharpness metrics are given. The central claim that FloTS "produces the most informative probabilistic forecasts" rests on a single qualitative case study (one "August" example, Fig. 2) and on a visual claim that calibration "places ground truth in higher probability regions."

The composite score normalizes across all models and horizons, so it is sensitive to which models are included. It also hides absolute differences. The heatmap values are not reproduced in the text, and there are no error bars, confidence intervals, or multiple seeds.

There is no persistence baseline and no climatological baseline. Persistence is notoriously hard to beat at 10–30 minute lead times for seeing, so without it the practical skill is unclear. Comparisons with prior ML seeing forecasters (e.g., Cherubini et al. 2021 on the same site) are absent.

The GP uses only a 500-sample subset and a single exponential kernel, with hyperparameters averaged across batches. This is a weak, arguably strawman probabilistic baseline. The abstract's statement that "deep learning approaches outperform standard recurrent networks" is oddly phrased, since LSTM is itself a recurrent network.

Calibration is fit on validation data with a bi-quadratic temperature function, but calibration is shown only on an example, with no held-out calibration evidence. The text is also internally odd: the uncalibrated density is described as "underconfident" while the calibration is said to sharpen it, yet a single example cannot validate this.

Train/validation/test splits are not described. Temporal leakage and seasonal stratification are not addressed. The data are linearly interpolated onto a 10-minute grid, which can smooth the high-frequency variability being forecast.

Notably, the paper's own finding is that deterministic LSTM beats FloTS on accuracy, and FloTS uses an LSTM encoder. The probabilistic advantage is asserted rather than quantified.

Potential Impact

The practical audience is observatory operations (queue scheduling, AO planning) and FSO communications. Probabilistic seeing nowcasts are a sensible direction, and a well-calibrated flow-based forecast could support risk-aware scheduling. However, this paper offers only a preliminary demonstration at one site, with univariate input only (no meteorological covariates), and no operational evaluation such as decision-value analysis. The impact will likely be modest: a citation within the optical turbulence forecasting community, and perhaps a stepping stone to the promised multi-variate and multi-site follow-ups. It is unlikely to change methodology outside this niche.

Timeliness & Relevance

Interest in ML-based turbulence forecasting is growing, driven by ELT-era facilities, queue-scheduled observing, and FSO and satellite links. Uncertainty quantification in forecasting is a current theme. The paper fits this trend but does not address a hard bottleneck. The problem of seeing forecasting has partly been tackled through numerical models (e.g., Meso-NH/ALTA) and ML.

Strengths

Clear motivation for moving from point to probabilistic forecasts.

Long (15-year) real dataset from a premier site.

Inclusion of a modern flow-based probabilistic model and a post-hoc calibration step.

Clean, readable presentation and a logical structure.

Honest conclusion that LSTM remains the most accurate deterministic model.

Limitations

Missing probabilistic metrics despite the claim, so the main thesis is not quantitatively supported.

No persistence or other simple baselines, and no comparison with the prior literature.

Weak GP baseline.

Single site, single variable, no covariates.

Qualitative calibration evidence from one example.

Limited reproducibility: no code or data availability statement, no splits, and no FloTS hyperparameters or flow depth.

Very short methods; most content is textbook description of RNN, LSTM, GP, and flow equations.

Preprocessing (interpolation, 10-minute resampling) is not evaluated for its effect on forecast difficulty.

Overall

This is a competent but thin application paper with incremental novelty and under-supported probabilistic claims. It is useful as a starting reference for probabilistic seeing forecasting and as a template for later, more rigorous work, but its standalone scientific impact is limited.

Rating:3.5/ 10

Significance 3Rigor 3Novelty 3Clarity 5.5

Generated Oct 22, 2026

Comparison History (0)

No comparisons yet.

Scorecard· 16/16

3.5/10 impact

Sensible but thin application benchmark whose headline probabilistic claims lack quantitative probabilistic metrics, with limited novelty and a niche audience.