Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • 13-curl
  • temporal-cleaning
  • 16-bracketed-forecasts
  • test-examples
  • progress-api
  • example
  • sphinx
  • newlock
9 results
Compare
Alan De Smet's avatar
Fix log levels/messages for failed filesets
Alan De Smet authored 
If all of the filesets are NOT expected, it's okay, just log at INFO.

If any fileset IS expected, that's worrying and generates a WARNING.
ERROR is too severe because we might be able to continue, and there is a
good chance that the error is entirely remote.
2ed504a6
History
Alan De Smet's avatar 2ed504a6
History
Name Last commit Last update
csppfetch
docs
example
.gitignore
COPYING.rst
LICENSE.rst
Makefile
README.rst
README.rst

csppfetch

csppfetch is a Python 3 module for downloading dynamic ancillary data. It can:

  • Maintain a local cache of ancillary data
  • Download the specific ancillary data files needed to process a given time step
    • Using the local cache if available
    • Optionally updating the local cache

Final files in the cache are atomically created; from the perspective of other programs a given ancillary data file either completely exists or doesn't exist at all.

Multiple copies of this module can safely run at the same time; locks are used to ensure they don't interfere with each other. (Your operating system and filesystem will need to correctly implement lockf.)

If a file is already present in the local cache, this module will NOT replace them; their existence is assumed to mean they are correct. To force a re-download, delete the existing file.

csppfetch implements the CSPP Geo ancillary download behavior guidelines.

Requires Python 3.6 or later.