yatsm.algorithms.postprocess module

Result post-processing utilities

Includes comission and omission tests and robust linear model result calculations

yatsm.algorithms.postprocess.commission_test(yatsm, alpha=0.1)[source]

Merge adjacent records based on Chow Tests for nested models

Use Chow Test to find false positive, spurious, or unnecessary breaks in the timeseries by comparing the effectiveness of two separate adjacent models with one single model that spans the entire time period.

Chow test is described:

\[\frac{[RSS_r - (RSS_1 + RSS_2)] / k}{(RSS_1 + RSS_2) / (n - 2k)}\]

where:

  • \(RSS_r\) is the RSS of the combined, or, restricted model
  • \(RSS_1\) is the RSS of the first model
  • \(RSS_2\) is the RSS of the second model
  • \(k\) is the number of model parameters
  • \(n\) is the number of total observations

Because we look for change in multiple bands, the RSS used to compare the unrestricted versus restricted models is the mean RSS values from all model.test_indices.

Parameters:
  • yatsm – fitted YATSM model to check for commission errors
  • alpha – significance level for F-statistic (default: 0.10)
Returns:

updated copy of yatsm.record with spurious models

combined into unified model

Return type:

np.ndarray

yatsm.algorithms.postprocess.omission_test(model, crit=0.05, behavior='ANY', indices=None)[source]

Add omitted breakpoint into records based on residual stationarity

Uses recursive residuals within a CUMSUM test to check if each model has omitted a “structural change” (e.g., land cover change). Returns an array of True or False for each timeseries segment record depending on result from statsmodels.stats.diagnostic.breaks_cusumolsresid.

Parameters:
  • crit – Critical p-value for rejection of null hypothesis that data contain no structural change
  • behavior – Method for dealing with multiple test_indices. ANY will return True if any one test index rejects the null hypothesis. ALL will only return True if ALL test indices reject the null hypothesis.
  • indices – Array indices to test. User provided indices must be a subset of model.test_indices.
Returns:

Array of True or False for each record where

True indicates omitted break point

Return type:

np.ndarray

yatsm.algorithms.postprocess.refit_record(model, prefix, estimator, fitopt=None, keep_regularized=False)[source]

Refit YATSM model segments with a new estimator and update record

YATSM class model must be ran and contain at least one record before this function is called.

Parameters:
  • model – YATSM model to refit
  • prefix – prefix for refitted coefficient and RMSE (don’t include underscore as it will be added)
  • estimator – instance of a scikit-learn compatible estimator object
  • fitopt – dict of options for the fit method of the estimator provided (default: None)
  • keep_regularized – do not use features with coefficient estimates that are fit to 0 (i.e., if using L1 regularization)
Returns:

updated model.record NumPy structured array with refitted

coefficients and RMSE

Return type:

np.array