Evaluating Machine Learning Forecasts of AI-Related Google Trends: Sparse Data and Predictive Uncertainty
An exploratory retrospective comparison of fourteen probabilistic forecasting procedures on six AI-related Google Trends indices, examining predictive uncertainty, sparse data and validation support.
Abstract
We evaluate fourteen probabilistic forecasting procedures on six artificial intelligence (AI)-related Google Trends indices using 48 four-week-ahead targets per query. For Agentic AI, Ridge records a mean ranked probability score (RPS) of 4.141 versus persistence’s 6.631, a 37.6% reduction, and mean absolute error (MAE) of 4.771 versus 7.521. However, its nominal 95% prediction intervals cover only 24 of 48 targets. For sparse Physical AI, random forest has the lowest observed RPS among the fourteen procedures (5.095), but intervals average 78 index points wide. Hurdle Beta covers all 45 zeros and none of the three positives without fitting either supervised component. For both sparse queries, tree validation uses only constant-response branches. No procedure leads across all six series. These exploratory findings connect score rankings to interval behavior, executed components and validation support; a restricted normalization analysis separately bounds earlier inputs compatible with retrospectively retrieved data.
Key Innovations
Joint Assessment of Forecast Scores and Model Execution
Compares fourteen procedures through predictive scores, interval behavior, target composition and records of the components actually fitted. Lower observed forecast loss is distinguished from adequate interval coverage.
Analysis of Choices Unresolved by Validation
Identifies exact tuning ties and untested fitted branches, then assesses equal-weight pools of tied empirical recency settings. Reproducible grid-order selection is not treated as evidence that validation discriminates among later fitted behaviors.
Conditional Identification of Earlier Inputs
Characterizes compatible earlier scales and rounded inputs under fixed-path normalization assumptions. These are deterministic compatibility bounds, not confidence intervals or a reconstruction of historical Google Trends releases.
Full Paper
Open PDF in a new tab ↗Research preprint dated 23 October 2026. The original 14-page PDF is reproduced without modification. If the embedded reader is unavailable, use the direct PDF link above.