Example Workflow¶
This section of the guide covers the process of running of the CCDC
implementation (yatsm.algorithms.ccdc.CCDCesque
), including
the change detection and classification steps of the process, on
an example dataset centered on Harvard Forest in Masssachusetts.
It assumes you have already installed YATSM.
Data¶
The data used in this example are part of the collection of time series subsets described in the landsat_stack repository.
Download at least the p013r030_images.tar.bz2 as described in the repository instructions. Place all downloads in the directory you want to work in (preferably a new directory), and then unzip the files. The directory structure created should look similar to the following:
$ tree
├── p013r030
│ ├── images
│ │ ├── example_img
│ │ ├── example_img.aux.xml
│ │ ├── example_img.hdr
│ │ ├── LE70130301999211EDC00
│ │ │ ├── LE70130301999211EDC00_MTL.txt
│ │ │ ├── LE70130301999211EDC00_stack
│ │ │ ├── LE70130301999211EDC00_stack.aux.xml
│ │ │ └── LE70130301999211EDC00_stack.hdr
...
│ │ ├── LT50130302011316GNC01
│ │ │ ├── LT50130302011316GNC01_MTL.txt
│ │ │ ├── LT50130302011316GNC01_stack
│ │ │ ├── LT50130302011316GNC01_stack.aux.xml
│ │ │ └── LT50130302011316GNC01_stack.hdr
│ │ └── YATSM
│ │ ├── yatsm_r0.npz
│ │ ├── yatsm_r100.npz
...
│ └── maps
│ ├── p013r030_HF_change_first_1985-2016.gtif
│ ├── p013r030_HF_change_first_1985-2016.gtif.aux.xml
│ ├── p013r030_HF_change_last_1985-2016.gtif
│ ├── p013r030_HF_change_last_1985-2016.gtif.aux.xml
...
│ └── p013r030_maps.qgs
├── p013r030_images.tar.bz2
├── p013r030_maps.tar.gz
└── p013r030_results.tar.gz
Setup¶
To use any of the built-in programs that facilitate running time series analysis, we need to describe the time series data using a CSV file and we need to provide our analysis parameters using a configuration file (specified in the YAML format).
To begin, copy the example configuration file located within this repository
as examples/p013r030/p013r030.yaml
, or by downloading it from Github
(p013r030.yaml).
Next, we will use the helper script located at scripts/gen_date_file.sh
to generate a CSV file with the columns date
and filename
and rows for each image we want to include in our analysis. For example,
date,filename
1984162,images/LT50130301984162XXX08/LT50130301984162XXX08_stack
1984274,images/LT50130301984274PAC08/LT50130301984274PAC08_stack
1984290,images/LT50130301984290PAC16/LT50130301984290PAC16_stack
You can use the Bash helper script, located in the repository as
scripts/gen_date_file.sh
, to create this CSV file. If this script
doesn’t work for your data, or you can’t run Bash scripts, generate
this information using some other means.
$ scripts/gen_date_file.sh -v images images.csv
Searching in images
Searching for pattern: L*stack
YYYYDOY starts at 9
Output file is images.csv
Found 423 images
Next, open the configuration file with a text editor and make some changes
under the dataset:
section to point to this newly generated images.csv
file (adjust this name if you named the CSV file something different). Change
the value of the input_file
parameter. You will also want to change the
values for output
and cache_line_dir
, which control the location
of output result files and “cached data” files, respectively.
dataset:
# Text file containing dates and images
input_file: "images.csv"
# Input date format
date_format: "%Y%j"
# Output location
output: "images/YATSM"
# Output file prefix (e.g., [prefix]_[line].npz)
output_prefix: "yatsm_r"
# Total number of bands
n_bands: 8
# Mask band (e.g., Fmask)
mask_band: 8
# List of integer values to mask within the mask band
mask_values: [2, 3, 4, 255]
# Valid range of band data
# specify 1 range for all bands, or specify ranges for each band
min_values: 0
max_values: 10000
# Use BIP image reader? If not, use GDAL to read in
use_bip_reader: False
# Directory location for caching dataset lines
cache_line_dir: "cache"
Pixel Plotter¶
A good first test to make sure your dataset configuration was successful,
as well as a good place to start for experimenting with different runtime
parameters, is using the yatsm pixel
command
Batch Change Detection Processing¶
The next step in the process is to run the change detection on all pixels in your dataset.
Change Detection Visualization¶
Classification Training¶
Find training data, remembering to record the range of time for which
the training data is accurate or relevant for so we will know which
time series segment attributes (X
) to match with the training data
labels (y
).
Classification Prediction¶
Using the trained classifier, classify the time series segments identified in the change detection step.