Causal Validation for Delayed Financial Targets

Research Status

Current production validation framework. This page also documents the August 2026 Kalman timing and covariance audit.

Financial validation becomes difficult when the target is delayed.

A forecast issued today for a long horizon cannot be fully scored tomorrow.

The model must preserve:

What it knew at issue time.
What mean it published.
What uncertainty it published.
Which future dates define the target.
When the target became fully observable.
Whether later model updates changed the state.

This page explains how Vyreon approaches that problem.

The Target Is A Horizon Average

Each maturity bucket contains several future sample dates.

The realized target is the average of the returns across that grid.

This is different from one terminal return.

The target reduces dependence on one exact endpoint, but it has a later maturity date.

The full target does not exist until the last sampled date has occurred.

Issue Date, Effective Horizon, And Maturity

Three dates must remain distinct.

Issue Date

The date on which the model forms and publishes an expectation.

Effective Horizon

The average location of the sampled future dates.

The public chart uses approximate effective horizons such as 19, 46, 90, and 243 days.

Full Maturity Date

The date by which the final sampled return is available.

The full maturity is closer to the upper boundary of the bucket.

A long-horizon target can have an effective date near 243 days while requiring up to 365 calendar days to mature completely.

Why Chart Alignment Matters

The public validation chart is plotted on an effective matured-date axis.

This makes the realized and expected horizon states easier to compare visually.

It does not mean the target was known on the effective date.

The chart footer must explain the axis definition.

Without that explanation, a reader could mistake an effective display date for the date on which the full label became observable.

Adaptive-Filter Validation

When a target matures, the live filter has a current state.

Adaptive-filter validation asks:

Was this newly matured label plausible under the filter state immediately before the label was assimilated?

The sequence is:

Retrieve the frozen feature vector from the original date.
Form the prior expectation.
Form the prior predictive uncertainty.
Compare the matured outcome with that prior.
Score the innovation.
Update the filter.

This is a valid model-health diagnostic.

It is not identical to scoring the exact interval that was published on the original issue date.

Strict Issue-Date Validation

Strict issue-date validation asks:

Did the outcome later fall inside the exact interval stored when the forecast was first issued?

This requires an immutable forecast ledger.

For each issue date and bucket, the ledger should preserve:

Model version.
Feature version.
Expected excess return.
Baseline.
Expected total return.
Predictive sigma.
Confidence interval.
Spot.
Issue timestamp.
Target definition.

Later, the fully matured target is joined back to that record.

Strict issue-date validation is the strongest test of the public forecast.

It is also operationally harder.

Why The Backend Matters

The telemetry backend preserves:

Raw email events.
Parsed payloads.
Alternate versions.
Canonical payload selection.
Prior rows.
Matured rows.
Derived outputs.
Receipt timestamps.

The latest valid payload by receipt time becomes canonical for a node and date.

Earlier payloads remain available for audit.

This supports correction and replay while preserving data lineage.

An immutable issue-date forecast table is the next logical validation layer.

The Original Kalman Sequence

The production estimator originally calculated:

prior mean
innovation
updated adaptive R using current innovation
predictive S using updated R
state update

The matured diagnostic later reconstructed interval width from a coefficient covariance that had already been updated by the same observation.

The reported row therefore mixed:

Prior mean.
Current-error-informed observation variance.
Posterior coefficient covariance.

The equations were numerically stable.

The timing was not one coherent prior forecast distribution.

Why The Issue Survived

The error did not cause:

NaNs.
Unstable coefficients.
Obviously wrong mean paths.
Poor average error.
Broken volatility correlation.

It mainly affected uncertainty bookkeeping.

The output looked conservative and smooth.

That made the defect difficult to notice.

Correct Prediction-First Order

The corrected sequence is:

Start from the posterior coefficient state after the previous observation.
Add process covariance to form P_prior = P + Q.
Freeze the observation variance available before the current innovation.
Calculate the prior expected value.
Calculate the innovation.
Calculate S = x' P_prior x + R_prior.
Score coverage and standardized innovation.
Update coefficients.
Update covariance using Joseph form.
Update the adaptive observation variance for the next observation.

This gives every scored row one consistent information set.

Joseph Covariance Update

The Joseph form is:

P_new
= (I - K H) P_prior (I - K H)'
+ K R K'

In the regression form, H is the feature vector.

The Joseph form is algebraically equivalent to the simpler covariance update under exact arithmetic.

It is more robust to floating-point asymmetry and loss of positive semidefiniteness.

The P0 Discovery

Correct prior scoring exposed a second hidden problem.

The coefficient covariance had been initialized as:

P0 = I

for a 14-dimensional standardized Ridge model.

That identity matrix was a diffuse numerical initialization, not an estimate of trained coefficient uncertainty.

On the first long-horizon scored row:

x' P0 x approximately 17.38

This produced a predictive sigma above 400 percent and a 95 percent half-width above 800 percent.

The old posterior-covariance reporting hid the problem because the first observation contracted P before the interval was recorded.

Trained Ridge Covariance

The production artifact already contained:

Ridge coefficients.
Ridge intercept.
Residual variance.
Scaler.

A coefficient covariance was derived from the same training problem.

For standardized design matrix X, residual variance R0, and Ridge penalty alpha, the candidate was:

P0
= R0 (X'X + alpha I)^-1
     X'X
     (X'X + alpha I)^-1

The matrix was exported per bucket.

The new artifacts were verified against production.

The following matched exactly:

Feature count.
Feature order.
Scalers.
Ridge coefficients.
Intercepts.
Residual variance.
Kalman hyperparameters.

The only substantive addition was the full per-bucket P0.

P0 Scale Comparison

The trained covariance traces were approximately:

Bucket	Trace of trained P0
8 to 30	3.68e-5
31 to 60	4.66e-5
61 to 120	3.64e-5
121 to 365	2.32e-5

The identity initialization had trace 14.

The diffuse identity covariance was hundreds of thousands of times larger in total variance.

Burn-In Verification

A full live-platform burn-in was run with:

Same production node identity.
Same model coefficients.
Same scaler.
Same residual variance.
Same process covariance.
Corrected timing.
Trained P0.

The startup logs confirmed that all four filters loaded the artifact matrices rather than the scalar fallback.

The pathological long-horizon startup interval disappeared.

Behavioral Comparison

The corrected burn-in produced:

Near-term average error near 1.29 percent.
Short-term average error near 1.26 percent.
Medium-term average error near 1.27 percent.
Long-term average error near 1.50 percent.
Coverage from approximately 98.7 to 99.5 percent across buckets.
Volatility correlation near 0.84 for close-to-close realized volatility and 0.81 for Parkinson volatility in that specific run.

The previous report had:

Similar mean paths.
Similar average errors.
Coverage near 98.1 to 98.5 percent.
Volatility correlation near 0.86 and 0.85.

The central model did not show a major step change.

The uncertainty model became slightly more conservative in the shorter buckets.

The startup covariance became much more realistic.

Interpreting 99 Percent Coverage

A nominal 95 percent interval covering approximately 99 percent of observations is conservative.

It is not automatically useless.

The important checks are:

Are intervals economically reasonable?
Are they stable?
Is average error stable?
Does the mean show bias?
Do misses still occur?
Is coverage inflated by startup pathologies?
Are observations serially dependent?
Is the diagnostic adaptive or strict origin-time?

The corrected run was not at 100 percent.

The long-horizon startup band was no longer absurd.

The result supports a conservative uncertainty model, not a trivial always-containing interval.

Remaining Validation Questions

Strict Origin-Time Coverage

The public issue-date interval should be scored directly from an immutable ledger.

Dependence

Horizon targets overlap heavily.

Coverage observations are not independent Bernoulli trials.

Interval Calibration

If narrower intervals are desired, Q, adaptive R, or an explicit calibration layer can be studied.

That is model refinement, not a reason to restore the inconsistent timing.

Live Accumulation

The corrected implementation should be monitored through new weekly maturations.

Multi-Asset Initialization

The trained P0 correction becomes more important when adding assets with different feature distributions or shorter histories.

Change-Control Lesson

The audit produced a new engineering rule:

No behavioral modification without a rollback point, an isolated artifact tag, and defined acceptance tests.

A core estimator change should include:

Code backup.
Artifact backup.
One-variable change.
Parity checks.
Full burn-in.
Fixed comparison dashboard.
Explicit accept or revert decision.

Mature-Once Processing

A delayed label should update the recursive model once.

The production system keeps a cursor for each symbol and bucket.

For each candidate origin date, it checks:

Does the frozen feature row exist?
Has the full upper horizon matured?
Can the horizon-grid return be calculated?
Is the causal baseline available?
Has this origin already been processed?

If the row is valid, it is scored and the cursor advances.

This prevents repeated assimilation of the same label.

A second scheduled daily pass once mutated state twice in an earlier version of the system. The duplicate pass was removed.

The lesson is that idempotency applies inside the model node as well as inside the backend.

Baseline Ordering

The causal baseline uses prior matured returns.

The current realized return is appended only after the baseline is computed.

The order is:

read prior label history
-> calculate rolling mean
-> calculate excess target
-> append current realized label

Appending first would let the target influence its own baseline.

That would shrink the measured excess return and contaminate validation.

Issued-Prior Covariance

The daily issued prior uses:

P_prior = P + Q

where P is the posterior covariance after the most recent matured update.

The feature-projected state variance is:

xPx = x' P_prior x

The full predictive variance is:

S = xPx + R_current

This matches the convention used when a later matured observation is scored.

Issued and matured uncertainty should describe the same type of prior distribution.

Process Covariance Relative To P0

The trained P0 matrices had traces near the same order as one step of total process covariance.

With 14 features and q_scale = 2.5e-6, the trace of Q was:

14 * 2.5e-6 = 3.5e-5

The trained P0 traces ranged approximately from 2.3e-5 to 4.7e-5.

This means the model allows one daily prediction step to add coefficient uncertainty comparable to the initial trained coefficient covariance.

That scale was preserved during the audit because changing Q at the same time would have prevented isolation of the P0 correction.

Whether Q is optimal remains a separate calibration question.

Coverage-Counter Distinction

The system has both stored matured rows and live summary counters.

Rows before the causal baseline becomes available can contain:

A realized return.
A prior-only interval.
No finite excess target.
No finite standardized innovation.

Such rows should not be counted as scored coverage failures.

The chart-generation notebook filters to finite scorable rows.

The internal coverage counter historically counted some pre-baseline rows.

That counter issue is separate from the chart's reconstructed coverage and should be corrected independently.

Burn-In As A Production Test

QuantConnect live warmup replays historical data backward from the current live date.

A full burn-in exercises:

Real option-chain availability.
Feature-engine warmup.
Frozen feature storage.
Target maturation.
Causal baseline growth.
KF initialization.
Recursive updates.
Backfill ledger behavior.
Shutdown diagnostics.

This is why an offline notebook cannot fully replace the live burn-in.

The platform is isolated and iteration is slow, but the test has high fidelity.

Current Conclusion

The August 2026 audit corrected a real uncertainty-timing issue and a latent covariance-initialization defect.

The corrected implementation:

Uses prior information consistently.
Preserves the full multidimensional Kalman state.
Starts from trained coefficient uncertainty.
Removes the pathological startup interval.
Retains stable expectation behavior.
Produces conservative but non-saturated coverage.

The next major validation advance is not another Kalman rewrite.

It is a strict issue-date forecast ledger.