A Neural ODE Can Be Physically Plausible and Still Miscalibrated

A learned differential equation can interpolate observed trajectories while getting the mechanism wrong. The weakness becomes visible under intervention: change an input more strongly than training did, suppress one component, or start from an unusual state. A second problem then appears. Even an ensemble that varies across fits may not widen its uncertainty enough where the intervention is unfamiliar.

This benchmark asks:

Can structural sign constraints and ensemble uncertainty make a learned neural vector field more honest when predicting unseen interventions on a bistable gene circuit?

The answer is mixed. Structure improves physical feasibility and slightly lowers trajectory error, but its uncertainty intervals are poorly calibrated under the held-out interventions.

Synthetic toggle-switch world

The data generator is a two-state mutually repressing switch:

\dot x=\frac{\alpha_x}{1+y^{n_y}}-d_xx+u_x(t), \qquad \dot y=\frac{\alpha_y}{1+x^{n_x}}-d_yy+u_y(t).

Production is positive, self-decay is negative, each state represses the other, and interventions $u_x,u_y$ have declared directions. Thirty training trajectories provide 1,350 state–intervention–derivative samples.

Two learned vector fields compete:

an unconstrained multilayer perceptron;
a structured production–decay network whose cross-repression, input effects, decay, and production terms are forced to the declared signs.

Six bootstrap members form each ensemble. Calibration uses only in-domain data; stronger pulses and knockout-like interventions are held out.

True and learned toggle-switch trajectories with uncertainty bands under held-out interventions. — **Figure 1.** Intervention rollouts. Bands are calibrated before the strong held-out perturbations; the structured model remains non-negative but often under-covers the truth.

Structure changes the hypothesis class

The unconstrained network can represent flexible local derivatives, including biologically implausible sign patterns. The structured model builds production and decay separately and constrains weights through positive transforms. This does not prove the true generator has been identified. It only excludes some vector fields known to violate the declared qualitative mechanism.

On held-out interventions, structured mean RMSE is $0.237$ , compared with $0.259$ for the unconstrained family. Its minimum prediction remains positive at $0.0146$ . The unconstrained model reaches $-1.178$ , an impossible concentration in the simulator.

True, unconstrained and sign-structured learned vector fields over the gene-circuit phase plane. — **Figure 2.** Phase-field comparison. Similar training loss can hide different global geometry; sign structure rules out some impossible flows but does not recover every separatrix correctly.

The error improvement is modest. The stronger result is feasibility: the structural architecture prevents negative rollouts in this benchmark. Yet feasibility alone is insufficient. A positive wrong trajectory is still wrong.

Calibration delivers the negative finding

Ensemble standard deviations are multiplied by a scale chosen on in-domain calibration trajectories. The unconstrained ensemble needs scale $3.52$ and reaches held-out coverage $0.983$ . The structured ensemble needs a much larger scale, $39.57$ , yet reaches only $0.721$ coverage.

Coverage, error, uncertainty association and abstention curves for two neural ODE families. — **Figure 3.** Calibration diagnostics. Lower RMSE and better physical signs do not imply valid predictive intervals under intervention shift.

Why can a structured ensemble be overconfident? All members share the same restrictive architecture and similar training support. They can agree with one another while sharing the same extrapolation bias. Bootstrap spread measures variation induced by resampling; it does not automatically represent uncertainty about missing mechanisms or intervention shift.

The structured error–uncertainty correlation is $0.514$ , whereas the unconstrained value is $0.953$ . Removing high-uncertainty cases reduces structured RMSE from $0.237$ to about $0.198$ , so its uncertainty is not useless. It is simply not calibrated enough to be interpreted as a valid interval.

Solver error is checked separately

Neural ODE predictions combine vector-field error with numerical integration error. Tightening solver tolerance reduces relative trajectory difference from about $9.55\times10^{-4}$ at $10^{-3}$ tolerance to $3.09\times10^{-6}$ at $2\times10^{-6}$ . This places integration error well below the reported model RMSE.

That separation matters. Changing the ODE solver cannot repair a learned vector field that extrapolates incorrectly; conversely, a good learned field can look poor if integrated too coarsely.

What “intervention-calibrated” should require

A scientifically useful interval under intervention should be evaluated on the intervention family of interest, not only observational trajectories. Ideally the protocol would:

declare intervention classes before training;
separate weak calibration interventions from stronger test interventions;
report marginal and trajectory-wise coverage;
examine interval width and failure localization;
include misspecified-mechanism simulations;
and allow abstention when support is insufficient.

The present title describes the goal, while the result shows it has not been fully achieved. That negative finding is more informative than quietly reporting only RMSE.

Claim boundary

All trajectories are synthetic. The learned weights do not identify a real gene circuit. The supported conclusion is:

In this toggle-switch benchmark, sign structure lowers mean held-out RMSE and prevents negative predictions, but the bootstrap ensemble remains severely under-covered under strong interventions.

Structure and calibration are separate design problems. A future model might combine sign constraints with mechanism ensembles, conformal calibration over intervention families, explicit epistemic priors, or a reject option.

References

R. T. Q. Chen et al., “Neural ordinary differential equations,” NeurIPS, 2018. Proceedings.
P. Kidger et al., “Neural controlled differential equations for irregular time series,” NeurIPS, 2020. Proceedings.
Y. Gal and Z. Ghahramani, “Dropout as a Bayesian approximation,” ICML, 2016. Proceedings.
A. Guo et al., “On calibration of modern neural networks,” ICML, 2017. Proceedings.