Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • 13-curl
  • temporal-cleaning
  • 16-bracketed-forecasts
  • test-examples
  • progress-api
  • example
  • sphinx
  • newlock
9 results
Compare
Alan De Smet's avatar
Bug fixes from last
Alan De Smet authored 
Bail early if all files present. Essential to avoid errors about
downloads being disabled when required files are present.

Ignore errors when mirroring; logging is all we need to do.
0f6992a2
History
Alan De Smet's avatar 0f6992a2
History
Name Last commit Last update
csppfetch
docs
example
.gitignore
Makefile
README.rst
README.rst

csppfetch

csppfetch is a Python 3 module for downloading dynamic ancillary data. It can:

  • Maintain a local cache of ancillary data
  • Download the specific ancillary data files needed to process a given time step
    • Using the local cache if available
    • Optionally updating the local cache

Final files in the cache are atomically created; from the perspective of other programs a given ancillary data file either completely exists or doesn't exist at all.

Multiple copies of this module can safely run at the same time; locks are used to ensure they don't interfere with each other. (Your operating system and filesystem will need to correctly implement lockf.)

If a file is already present in the local cache, this module will NOT replace them; their existence is assumed to mean they are correct. To force a re-download, delete the existing file.

csppfetch implements the CSPP Geo ancillary download behavior guidelines.

Requires Python 3.6 or later.