The Direction-Execution Gap in Physics-Informed Neural Network Optimization
An optimizer-aware reliability audit for detecting when a locally favorable PINN update deteriorates after the complete finite optimizer step is executed.
PINN optimization is often interpreted through component-gradient geometry, but training acts through a finite, optimizer-state-dependent displacement. We introduce the direction-execution gap: a component predicts local descent along the displacement that is ultimately executed, yet its loss increases after that displacement is completed. The proposed fixed-set audit evaluates the actual optimizer map on identical points before and after execution, isolating parameter-induced change from collocation replacement. In 111 paired runs across four PDE families and three update rules, first-order sign accuracy was 86.39% and reversal prevalence 13.56%. One directional Hessian-vector product raised sign accuracy to 99.63% and reduced the median normalized magnitude-error ratio to 1.284 × 10−3, a 99.87% reduction. These results establish a directly measurable reliability object for the complete optimizer map and show that local directional promise can diverge from realized component progress at practical PINN step lengths. A 30-run curvature-aware acceptance study reduced median reversal prevalence from 11.61% under Adam to 3.44%, while final-error comparisons remained statistically indistinguishable from matched controls. This establishes executed-step reliability as an actionable objective complementary to solver accuracy.
