Skip to content
Snippets Groups Projects
Commit c9399bc4 authored by Alan De Smet's avatar Alan De Smet
Browse files

Moved nearly everything into GitLab issues.

parent 17412334
Branches
No related tags found
No related merge requests found
This should move into GitLab issues at some point. This should move into GitLab issues at some point.
- Some _combinations_ of tests generate ResourceWarnings; figure out why; fix or suppress
- Here's a relatively minimal example, but there are more warnings in a full 'make test'
- python3 -m csppfetch.test DownloaderTests.test_cache_complex DownloaderTests.test_cache_simple ExclusiveLockFileTests.test_exclusivelockfile_handle
- Add missing tests
- DownloadStatistics.__add__ doctest and unittest
- human_bytes doctest and unittest
- when updating the cache, inside a given retry (assumption: that's the outer
loop!), track downloads:
- Successful - permanently drop from other FileSet lists, as they'll
generate misleading cache-hit statistics. Or somehow suppress? Should
statistics take a filename and don't include a given filename more than once?
- Failed - Don't retry?
- What other usage patterns are worth handling? - What other usage patterns are worth handling?
- "I need a pair of times around my target time" - Does anyone need this at - "I need a pair of times around my target time" - Does anyone need this at
all? all?
...@@ -22,19 +8,3 @@ This should move into GitLab issues at some point. ...@@ -22,19 +8,3 @@ This should move into GitLab issues at some point.
- Target time: 15:15Z - Target time: 15:15Z
- Result: 15:00Z AND 16:00Z - Result: 15:00Z AND 16:00Z
- "I need a pair of forecasts around my target time" - cspp-geo-aitf needs
GFS data exactly like this!
- Example:
- A set of forecasts is generated every 6 hours starting at 0:00Z.
The set of forecasts make predictions for every 3 hours out
- Target time: 15:15Z
- Result:
- Ideal: 12:00Z's +3 hour and +6 hour forecasts
- Fallback: 6:00Z's +9 hour and +12 hour forecasts
- Fallback: 6 hours earlier using 6 later later forecasts,
repeating until hitting 72 hour forecasts.
- Should csppfetch.Downloader.download_for_time and csppfetch.Downloader.update_cache automatically construct and return a DownloadStatistics object? Simplifies API a little bit. DownloadStatistics would need to store a list of downloaded files for download_for_time's caller to use, but maybe that's a good idea anyway? If so, becomes a logical place for progress information (see "add a progress callback" below").
- Add a progress callback
- It's silly for csppfetch to log progress to logging.progress if available and logging.info if not. csppfetch.Downloader.download_for_time and csppfetch.Downloader.update_cache should take a callback which will be called occasionally with useful information and a copy of the DownloadStatistics if not-null.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment