conda
environment
Creating the To be able to run the local processing script, we need to install the following packages into whichever conda
python environment we are using:
pip install -i https://sips.ssec.wisc.edu/eggs -U flo rados
pip install -i https://sips.ssec.wisc.edu/eggs -U sipsprod glutil
pip install -i https://sips.ssec.wisc.edu/eggs -U timeutil
pip install -i https://sips.ssec.wisc.edu/eggs -U simple
After package delivery we can access package information from ipython
using the delivered_software
method of the
flo.sw.lib.glutil
module:
from flo.sw.lib.glutil import delivered_software
delivered_software.lookup('fusion_matlab', delivery_id='20180914-1')
which gives the output
Delivery(id='20180914-1', name='fusion_matlab', version='1.2.2', path='/mnt/deliveredcode/deliveries/fusion_matlab/20180914-1')
Local Deployment of python glue code
The glue code for fusion_matlab
, ~/code/PeateScience/packages/fusion_matlab/source/flo/__init__.py
, can be linked into the local
execution directory ~/code/PeateScience/local/dist/fusion_matlab
:
cd ~/code/PeateScience/local/dist/fusion_matlab
ln -s ../../../packages/fusion_matlab/source/flo ./
Running the local Fusion processing script
We begin by setting some directories and importing the local execution script:
repo=$HOME'/git'
work_dir='/data/'$USER'/fusion_matlab/work/local_processing'
str_import_fusion_matlab="import os; from timeutil import datetime, timedelta, TimeInterval; os.chdir('$repo/fusion_matlab'); import example_local_prepare; os.chdir('$work_dir')"
We can then test the generating of valid contexts using print_contexts()
:
satellite='snpp'
python -c "$str_import_fusion_matlab; granule = datetime(2014, 10, 15, 8, 42); interval = TimeInterval(granule, granule+timedelta(minutes=60)); example_local_prepare.print_contexts(interval, '$satellite', '1.0dev6')"
resulting in
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 8, 42)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 8, 48)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 8, 54)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 0)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 6)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 12)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 18)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 24)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 30)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 36)}
{'satellite': 'snpp', 'version': '1.0dev6', 'granule': datetime.datetime(2014, 10, 15, 9, 42)}
When we are confident that the fusion_matlab
glue code is working correctly, we can actually run the package"
python -c "$str_import_fusion_matlab; granule = datetime(2014, 10, 15, 8, 42); interval = TimeInterval(granule, granule+timedelta(minutes=0)); example_local_prepare.local_execute_example(interval, '$satellite', '1.0dev6', skip_prepare=False, skip_execute=True)"
Running the local Fusion Quicklooks processing script
We begin by setting some directories and importing the local execution script:
repo=$HOME'/git'
work_dir='/data/'$USER'/fusion_matlab/work/local_processing'
str_import_fusion_matlab_ql="import os; from timeutil import datetime, timedelta, TimeInterval; os.chdir('$repo/fusion_matlab'); import example_local_prepare_ql; os.chdir('$work_dir')"
We can then test the generating of valid contexts using print_contexts()
:
satellite='snpp'
python -c "$str_import_fusion_matlab_ql; granule = datetime(2014, 10, 15, 8, 42); interval = TimeInterval(granule, granule+timedelta(days=5)); example_local_prepare_ql.print_contexts(interval, '$satellite', '1.0dev4')"
resulting in
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 15, 0, 0)}
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 16, 0, 0)}
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 17, 0, 0)}
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 18, 0, 0)}
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 19, 0, 0)}
{'satellite': 'snpp', 'version': '1.0dev4', 'granule': datetime.datetime(2014, 10, 20, 0, 0)}
When we are confident that the fusion_matlab
glue code is working correctly, we can actually run the package"
python -c "$str_import_fusion_matlab_ql; granule = datetime(2014, 10, 15, 0); interval = TimeInterval(granule, granule+timedelta(days=0)); example_local_prepare_ql.local_execute_example(interval, '$satellite', '1.0dev4', skip_prepare=False, skip_execute=True)"