... | ... | @@ -6,8 +6,7 @@ We import the `flo3` interface python code for `fusion_matlab` into the software |
|
|
```bash
|
|
|
sudo su - flo
|
|
|
cd /mnt/software/geoffc
|
|
|
mv fusion_matlab fusion_matlab_old
|
|
|
rsync -urLv /home/geoffc/code/PeateScience/local/dist/fusion_matlab . --progress --exclude=.*.sw*
|
|
|
rsync -urLv /home/geoffc/py3flo/git/fusion_matlab/source/flo/ fusion_matlab/flo/ --progress --exclude=.*.sw*
|
|
|
```
|
|
|
### Deploy the glue code to the development (flo) account cluster
|
|
|
We import the `flo3` interface python code for `fusion_matlab` into the software tree
|
... | ... | @@ -15,18 +14,16 @@ We import the `flo3` interface python code for `fusion_matlab` into the software |
|
|
```bash
|
|
|
sudo su - flo
|
|
|
cd /mnt/software/flo/
|
|
|
mv fusion_matlab fusion_matlab_old
|
|
|
rsync -urLv /home/geoffc/code/PeateScience/local/dist/fusion_matlab . --progress --exclude=.*.sw*
|
|
|
rsync -urLv /home/geoffc/py3flo/git/fusion_matlab/source/flo/ fusion_matlab/flo/ --progress --exclude=.*.sw*
|
|
|
```
|
|
|
|
|
|
### Commit glue code to PeateScience repo
|
|
|
The actual glue code was copied to `/mnt/software` in the last step, but pushing the `fusion_matlab` python code to the `PeateScience` repo will provide the submission scripts
|
|
|
The actual glue code was copied to `/mnt/software` in the last step, but pushing the python code to the `fusion_matlab` repo will provide the submission scripts
|
|
|
`example_local_prepare.py` and `submit_fusion_matlab.py` for use on condor.
|
|
|
```bash
|
|
|
cd ~/code/PeateScience
|
|
|
cd /home/geoffc/py3flo/git/fusion_matlab
|
|
|
git pull
|
|
|
git add ~/code/PeateScience/packages/fusion_matlab
|
|
|
git commit fusion_matlab -m "Initial commit of the fusion_matlab package."
|
|
|
git commit -m "Some change in the glue code."
|
|
|
git push
|
|
|
```
|
|
|
|
... | ... | @@ -37,13 +34,19 @@ sudo su - flo |
|
|
cd /data/geoffc/fusion_matlab/cluster_processing/logs
|
|
|
|
|
|
$ python /home/geoffc/code/PeateScience/packages/fusion_matlab/submit_fusion_matlab.py
|
|
|
(INFO):submit_fusion_matlab.py:<module>:30: Submitting intervals...
|
|
|
(INFO):submit_fusion_matlab.py:<module>:32: Submitting interval 2015-04-17 14:36:00 -> 2015-04-17 14:36:59
|
|
|
(INFO):submit_fusion_matlab.py:<module>:36: There are 1 contexts in this interval
|
|
|
{'satellite': 'snpp', 'version': '1.0dev0', 'granule': datetime.datetime(2015, 4, 17, 14, 36)}
|
|
|
(INFO):submit_fusion_matlab.py:<module>:42: First context: {'satellite': 'snpp', 'version': '1.0dev0', 'granule': datetime.datetime(2015, 4, 17, 14, 36)}
|
|
|
(INFO):submit_fusion_matlab.py:<module>:43: Last context: {'satellite': 'snpp', 'version': '1.0dev0', 'granule': datetime.datetime(2015, 4, 17, 14, 36)}
|
|
|
(INFO):submit_fusion_matlab.py:<module>:44: xrange(86694864, 86694865)
|
|
|
Verbosity is 2
|
|
|
(INFO): Submitting intervals...
|
|
|
(INFO): Submitting interval 2018-04-10 00:00:00 -> 2018-04-10 23:59:59
|
|
|
(INFO): Opening log file fusion_matlab_snpp_s201804100000_e201804102359_c20201124170636.log
|
|
|
(INFO): There are 240 contexts in this interval
|
|
|
(INFO): First context: {'satellite': 'snpp', 'version': '1.1.0dev4', 'granule': datetime.datetime(2018, 4, 10, 0, 0)}
|
|
|
(INFO): Last context: {'satellite': 'snpp', 'version': '1.1.0dev4', 'granule': datetime.datetime(2018, 4, 10, 23, 54)}
|
|
|
(INFO): Attempted 240 submits; results: 239/1/0 (submitted/not needed/not ready) [jobs 187647274 to 187647512]
|
|
|
(INFO): Job nums: range(187647274, 187647513)
|
|
|
(INFO): contexts: [{'satellite': 'snpp', 'version': '1.1.0dev4', 'granule': datetime.datetime(2018, 4, 10, 0, 0)}, {'satellite': 'snpp', 'version': '1.1.0dev4', 'granule': datetime.datetime(2018, 4, 10, 23, 54)}]; job numbers: [187647274,187647512]
|
|
|
(INFO): job numbers: [187647274..187647512]
|
|
|
|
|
|
(INFO): Closing log file fusion_matlab_snpp_s201804100000_e201804102359_c20201124170636.log
|
|
|
```
|
|
|
We can keep track of running jobs by doing the various incantations:
|
|
|
```bash
|
... | ... | |