Skip to content
Snippets Groups Projects
Verified Commit cb58f89a authored by David Hoese's avatar David Hoese
Browse files

Add bash backfill one liner to README

parent 68d0020b
No related branches found
No related tags found
No related merge requests found
......@@ -168,4 +168,17 @@ performance.
For the Buoy instrument there are typically more than one ascii file
(metdata and limnodata). Both of these files should be added before the
averaging command is run.
\ No newline at end of file
averaging command is run.
A bash one-liner for processing one year and running averages 1 month at a time
(to avoid overwhelming the database):
.. code-block:: bash
# AOSS Tower
for YEAR in `seq 2023 -1 2006`; do echo ${YEAR}; time find /data1/raw/aoss/tower/${YEAR} -name "*.ascii" -print0 | sort -z | xargs -r0 -n1 /opt/metobs/aoss_tower/bin/python -m aosstower.level_00.influxdb -v --sleep-interval 5.0 --bulk 10000 --influxdb-token $(</home/metobs/influxdbv2_metobs_rw.txt); for month in `seq -f %02.0f 1 12`; do /opt/metobs/aoss_tower/bin/python -m metobscommon.influxdb -vvv --influxdb-token $(</home/metobs/influxdbv2_metobs_rw.txt) run_manual_average --stations aoss.tower -s ${YEAR}-${month}-01T00:00:00 -e $(date -d "${YEAR}-${month}-01 + 1 month" +%Y-%m-%dT%H:%M:%S); sleep 5.0; done; echo "Done with $YEAR"; sleep 60; done
# Mendota Buoy
for YEAR in `seq 2023 -1 2013`; do echo ${YEAR}; time find /data1/raw/mendota/buoy/${YEAR} \( -name "*limnodata.*ascii" -o -name "*metdata.*.ascii" \) -print0 | sort -z | xargs -r0 -n1 /opt/metobs/mendota_buoy/bin/python -m mendotabuoy.level_00.influxdb -v --sleep-interval 5.0 --bulk 10000 --influxdb-token $(</home/metobs/influxdbv2_metobs_rw.txt); for month in `seq -f %02.0f 2 11`; do /opt/metobs/mendota_buoy/bin/python -m metobscommon.influxdb -vvv --influxdb-token $(</home/metobs/influxdbv2_metobs_rw.txt) run_manual_average --stations mendota.buoy -s ${YEAR}-${month}-01T00:00:00 -e $(date -d "${YEAR}-${month}-01 + 1 month" +%Y-%m-%dT%H:%M:%S); sleep 5.0; done; echo "Done with $YEAR"; sleep 60; done
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment