SPARChive
Tools and scripts used in the SSEC Portable Atmospheric Research Center (SPARC). The SPARC is a trailer that contains various instruments for measuring atmospheric conditions during field experiments. All the instruments are connected to an archive computer, SPARChive. Data is pulled or collected from the instruments and archived on the SPARChive machine. The scripts handling this part of data ingest are collectively called "archive" scripts. Optionally, scripts wait for data to be written to the archive location and then perform various actions. Primarily these scripts are used to push data to SSEC servers for further processing or visualization. These scripts are collectively called "push" scripts.
This repository also includes software used on the SSEC archive server for managing data files. It also contains the web application software for a URL API to the SSEC server's storage. See the Repository Structure section below.
Current SPARC Instruments:
- Atmospheric Emitted Radiance Interferometer (aeri)
- Campbell Datalogger - Surface Meteorological Observation System (surfacemet)
- High Spectral Resolution LiDAR (hsrl)
- Radiosondes (radiosondes)
- HALO Photonics Stream Line Wind-Pro LiDAR (windpro)
Developed by: David Hoese, SSEC
Archive Scripts
TODO
Push Scripts
TODO
Repository Structure
-
- remote_scripts: Scripts used to start processing on remote servers. Scripts that don't fit anywhere else. WARNING:
- This directory may be removed in the future.
- router_configs: Router and modem backup configurations and scripts.
- scripts: Task scripts for the sparchive machine. Usually the scripts run via the "sparcctl" command.
-
- sparc_cms: The Django/Mezzanine-based web application for the SPARC site. This code makes up the bones of the site to
- provide a CMS interface.
-
- sparchive: Python package for running everything. Includes SSEC server archive code, instrument utilities that run
- on sparchive, and anything else that is python and deals with instrument data.
- sparchive_api (future): Pyramid web application to provide a URL API to the SSEC server's archive storage.
- sparcler_scripts: Script that run on the quicklooks laptop, SPARCler.
- circus.ini: Circus daemon configuration file to be place on "sparchive:/etc/".
- circus_init.conf: Simple configuration file to get circus in UpStart's init system. May be removed in the future.
- init.sh: Script to be loaded from sparc accounts ".bashrc" on sparchive before running anything.
- sparchive_backup.sh/sparchive_restore.sh: Simple transfer scripts to jumpdrive for sparchive backup/restore.