Set up Sphinx doc generation

README.rst

-csppfetch
-=========
+csppfetch Introduction
+----------------------
 
-csppfetch is a Python 3 module for downloading dynamic ancillary data. It
-supports downloading individual files needed for processing a given time step
-(and optionally maintaining a local cache as it does) as well as explicitly
-maintaining a local cache. It uses fcntl.lockf to allow multiple copies to
-safely run at the same time.  It implements the 
-`CSPP Geo ancillary download behavior guidelines <https://docs.google.com/document/d/1phQRbELddAl4AG2EdAvzlJOKdYD0HRu-TpD3fj7J1rA/edit?pli=1>`_.
-
-Requires Python 3.6 or later.
-
-
-Example
--------
-
-Setup
-
->>> import csppfetch
->>> import datetime
->>> sst_filename = "avhrr-only-v2.%Y%m%d_preliminary.nc"
->>> sst = csppfetch.Downloader(
-...     name = "Sea Surface Temperature",
-...     package_env_id = 'CSPP_GEO_AITF_',
-...     url_base = "https://geodb.ssec.wisc.edu/ancillary/",
-...     url_relative = "%Y_%m_%d_%j/"+sst_filename,
-...     local = sst_filename,
-...     period = datetime.timedelta(hours=24),
-...     epoch_start = datetime.datetime(2010,1,1,3,0,0)
-...     )
-
-(Some imports for these examples, not required for module use!)
- 
->>> from tempfile import TemporaryDirectory
->>> import os
-
-
-Download files needed to process a scan at 2019-5-27 12:00:00:
-
->>> scan_time = datetime.datetime(2019,5,27, 12,0,0)
->>> with TemporaryDirectory() as example_dir:
-...     sst.download_for_time(scan_time, example_dir)
-...     os.path.exists(example_dir+"/avhrr-only-v2.20190527_preliminary.nc")
-True
+csppfetch is a Python 3 module for downloading dynamic ancillary data. It can:
 
-Download last 7 days of files
+- Maintain a local cache of ancillary data 
+- Download the specific ancillary data files needed to process a given time step
+    - Using the local cache if available
+    - Optionally updating the local cache
 
->>> with TemporaryDirectory() as example_dir:
-...     sst.mirror(example_dir)
-...     len(os.listdir(example_dir)) > 3
-True
+Final files in the cache are atomically created; from the perspective of other
+programs a given ancillary data file either completely exists or doesn't exist
+at all.
 
-(The test is ">3" because due to vaguaries of mirroring, we are likely to get fewer, sometimes many fewer, than the ideal 6 or 7.)
+Multiple copies of this module can safely run at the same time; locks are used
+to ensure they don't interfere with each other. (Your operating system and
+filesystem will need to correctly implement ``lockf``.)
 
+If a file is already present in the local cache, this module will NOT replace
+them; their existence is assumed to mean they are correct. To force a
+re-download, delete the existing file.
 
-
-Usage
------
-
-See ``import csppfetch; help(csppfetch.Downloader.__init__)`` for details on
-the arguments to Downloader.
-
-TODO: Expand on WHY you'd use various options.
-
-
-
-
-``csppfetch.Downloader.download_for_time`` implements the 
+csppfetch implements the 
 `CSPP Geo ancillary download behavior guidelines <https://docs.google.com/document/d/1phQRbELddAl4AG2EdAvzlJOKdYD0HRu-TpD3fj7J1rA/edit?pli=1>`_.
-It defaults to to abandoning a download after 30 seconds, will retry 3
-failures, and waits 20 seconds between retry attempts. These can be overridden
-with the environment variables ``<prefix>_TIMEOUT``, ``<prefix>_RETRIES``, and
-``<prefix>_RETRY_WAIT``, where ``<prefix>`` is whatever is specified by
-``package_env_id`` in the ``csppfetch.Downloader`` constructor. ``packages_env_id``
-should probably end in "_ANCIL_" to best match the guidelines, so you might use
-``package_env_id="CSPP_GEO_AITF_ANCIL_"`` instead of ``package_env_id="CSPP_GEO_AITF"``. Similarly,
-you can replace the default URL in ``url_base`` with a comma seperated list using
-the ``<prefix>_URL`` environment variable.
-
-By default download_for_time seeks the nearest preceeding time relative to the
-time passed in.
-
-
-
 
+Requires Python 3.6 or later.

docs/.gitignore

+.*.swp
+sphinx-build
+sphinx-env
+_build

docs/Makefile

+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= ./sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help: $(SPHINXBUILD)
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile $(SPHINXBUILD)
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+sphinx-build:
+	./setup-sphinx

docs/api.rst

+API
+===
+
+
+.. automodule:: csppfetch
+   :members:

docs/conf.py

+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+sys.path.insert(0, os.path.abspath('..'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = 'csppfetch'
+copyright = '2021, Alan De Smet'
+author = 'Alan De Smet'
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [ 'sphinx.ext.autodoc', 'sphinx.ext.doctest']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', 'sphinx-build', 'sphinx-env']
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'alabaster'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+#html_static_path = ['_static']

docs/index.rst

+csppfetch
+=========
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   api
+
+
+.. include:: ../README.rst
+
+..  XXXTemporarily Disabled
+    Indices and tables
+    ==================
+
+    * :ref:`genindex`
+    * :ref:`modindex`
+    * :ref:`search`
+
+
+
+
+Examples
+========
+
+Setup
+
+>>> import csppfetch
+>>> import datetime
+>>> sst_filename = "avhrr-only-v2.%Y%m%d_preliminary.nc"
+>>> sst = csppfetch.Downloader(
+...     name = "Sea Surface Temperature",
+...     package_env_id = 'CSPP_GEO_AITF_',
+...     url_base = "https://geodb.ssec.wisc.edu/ancillary/",
+...     url_relative = "%Y_%m_%d_%j/"+sst_filename,
+...     local = sst_filename,
+...     period = datetime.timedelta(hours=24),
+...     epoch_start = datetime.datetime(2010,1,1,3,0,0)
+...     )
+
+(These imports are for these examples. They're not required to use the module!)
+ 
+>>> from tempfile import TemporaryDirectory
+>>> import os
+
+
+Download files needed to process a scan at 2019-5-27 12:00:00:
+
+>>> scan_time = datetime.datetime(2019,5,27, 12,0,0)
+>>> with TemporaryDirectory() as example_dir:
+...     sst.download_for_time(scan_time, example_dir)
+...     os.path.exists(example_dir+"/avhrr-only-v2.20190527_preliminary.nc")
+True
+
+Download last 7 days of files
+
+>>> with TemporaryDirectory() as example_dir:
+...     sst.mirror(example_dir)
+...     len(os.listdir(example_dir)) > 3
+True
+
+(The number of files is compared to 3 because due to vaguaries of creation and
+mirroring, we are likely to get fewer, sometimes many fewer, than the ideal 6
+or 7.)
+
+
+
+Usage
+=====
+
+See ``import csppfetch; help(csppfetch.Downloader.__init__)`` for details on
+the arguments to Downloader.
+
+TODO: Expand on WHY you'd use various options.
+
+
+
+
+``csppfetch.Downloader.download_for_time`` implements the 
+`CSPP Geo ancillary download behavior guidelines <https://docs.google.com/document/d/1phQRbELddAl4AG2EdAvzlJOKdYD0HRu-TpD3fj7J1rA/edit?pli=1>`_.
+It defaults to to abandoning a download after 30 seconds, will retry 3
+failures, and waits 20 seconds between retry attempts. These can be overridden
+with the environment variables ``<prefix>_TIMEOUT``, ``<prefix>_RETRIES``, and
+``<prefix>_RETRY_WAIT``, where ``<prefix>`` is whatever is specified by
+``package_env_id`` in the ``csppfetch.Downloader`` constructor. ``packages_env_id``
+should probably end in "_ANCIL_" to best match the guidelines, so you might use
+``package_env_id="CSPP_GEO_AITF_ANCIL_"`` instead of ``package_env_id="CSPP_GEO_AITF"``. Similarly,
+you can replace the default URL in ``url_base`` with a comma seperated list using
+the ``<prefix>_URL`` environment variable.
+
+By default download_for_time seeks the nearest preceeding time relative to the
+time passed in.
+
+
+
+
+

docs/make.bat

+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=.
+set BUILDDIR=_build
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.http://sphinx-doc.org/
+	exit /b 1
+)
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd