Skip to content

Improve data fetching

Em Zhan requested to merge improve-data-fetching into main

Makes it easier to get the required data files needed to regrid by introducing an init function for static ancillary files and integrating asipscli for dynamic ancillary files.

The new CLI now looks like this:

Usage: sipsanc [OPTIONS] COMMAND [ARGS]...

  Transform ancillary products into a convenient format

Options:
  --version   Show the version and exit.
  -h, --help  Show this message and exit.

Commands:
  init    Initialize products for `sipsanc regrid`
  list    List all products and indicate which ones need to be initialized
  regrid  Regrid products to low Earth orbit satellite pixels
  clean   Remove files and directories created by `sipsanc init`

Using inline script metadata, it was possible to remove the pyhdf dependency from the sips-anc package while ensuring one-time run fetch script still works. The current development plan to run the one-time fetch scripts on sipsdev, then local can get those files via sipsanc init all --from sipsdev. The default source --from release is not yet implemented and will need be considered as part of the release process for this package.

Merge request reports