After a Good Fit, What Should We Measure?
A short note on choosing the next observation when several parameter explanations already fit the data.
Topic
Data & AI
Learning from data while testing what can be inferred.
Start with an article
Filter writing on this topic →Choose the Question Before the Score
A small error or an improved average answers only its own question. Event timing and worst-group reliability need their own checks.
Did the Hotspot Move, or Did It Grow?
Two spatial maps can share a convincing least-cost explanation without revealing the history that produced them. A controlled unbalanced optimal transport study separates endpoint fit, numerical convergence and evidence for movement versus local growth.
If You Only Observe Aggregates, Can You Still Bound the Hidden Epidemic?
Delayed regional totals can support a precise-looking estimate without tightly constraining the hidden state. A six-region synthetic SIR benchmark compares ensemble intervals with validated set bounds, including the cost of bounds that become almost useless.
When Early Warnings Cannot Tell the Difference
A system that recovers slowly and an environment that persists can leave the same statistical signature. An exactly sampled stochastic experiment shows what passive monitoring can identify, and which extra observations change the question.
Assimilating What You Can Observe
A frozen six-patch EnKF benchmark shows that an explicit aggregate-and-delay observation operator improves synthetic state and peak forecasts, yet undercovers the latent state and therefore earns only a PARTIAL conclusion.
Follow a reading path
All paths →When can we trust a model?
From a convincing fit to a useful guarantee: ask what the observations, evaluation and bounds actually establish.
First: When the Fit Is Not the Model
Go a little deeper
Project overviews
2026
CUT-AUDIT-ER: When the Highest Score Is Not the Best Repair
A study of verification-only entity repair separates finding the highest score from choosing a useful repair, and fewer complete evaluations from lower total cost.
2026
Intervention-Calibrated Neural ODEs
Sign-structured and unconstrained neural vector fields are tested on strong held-out interventions in a synthetic toggle switch.
2026
Topology at the Edge of Resolution
A controlled false-threshold audit of melt-pond connectivity under blur, downsampling, segmentation error, finite windows, and morphology shift.
2026
Real-Time sEMG Conditioning with Tracked Harmonic Cancellation
A reproducible synthetic implementation separates causal sample processing from gated mains-frequency tracking for harmonic cancellation and envelope extraction.
2026
Assimilating What You Can Observe
A frozen six-patch EnKF benchmark shows that the declared aggregate-and-delay observation operator improves synthetic state and peak forecasts but undercovers the latent state, limiting the conclusion to PARTIAL.
2026
Bounding a Hidden Epidemic from Delayed Aggregates
A synthetic six-region SIR benchmark compares central ensemble intervals with validated constrained-box enclosures, separating conditional containment from useful precision and testing what remains predictable after observations stop.