... | ... | @@ -123,7 +123,7 @@ rados -p dev --id flo get flo3/91069111/VNP02FSN.A2015091.0000.001.2018025170339 |
|
|
## Running in Forward Stream
|
|
|
|
|
|
Job parameters for `FUSION_MATLAB` can be found from
|
|
|
```
|
|
|
```sql
|
|
|
flo3=> select * from forward_streams where name = 'FusionMatlab';
|
|
|
id | name | offset_start | offset_end | find_contexts_arguments | workflow_head | workflow_targets | workflow_download_onlies | job_mods | output_volume | num_retries | expiration
|
|
|
----+--------------+--------------+------------+-----------------------------------------------+------------------------------------+------------------------------------------------+--------------------------+-------------------------------+---------------+-------------+------------
|
... | ... | @@ -131,7 +131,7 @@ flo3=> select * from forward_streams where name = 'FusionMatlab'; |
|
|
(1 row)
|
|
|
```
|
|
|
and for `FUSION_MATLAB_QL`:
|
|
|
```
|
|
|
```sql
|
|
|
flo3=> select * from forward_streams where name = 'FusionMatlabDailyQL';
|
|
|
id | name | offset_start | offset_end | find_contexts_arguments | workflow_head | workflow_targets | workflow_download_onlies | job_mods | output_volume | num_retries | expiration
|
|
|
----+---------------------+--------------+------------+-----------------------------------------------+---------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------+-------------+------------
|
... | ... | @@ -139,7 +139,7 @@ flo3=> select * from forward_streams where name = 'FusionMatlabDailyQL'; |
|
|
(1 row)
|
|
|
```
|
|
|
To preview the submission of the Fusion Matlab level-1b files to Forward Stream, we enter the following in the `psql` shell:
|
|
|
```
|
|
|
```sql
|
|
|
explain INSERT INTO forward_streams (
|
|
|
name, offset_start, offset_end, find_contexts_arguments,
|
|
|
workflow_head, workflow_targets, job_mods
|
... | ... | @@ -156,7 +156,7 @@ VALUES ( |
|
|
;
|
|
|
```
|
|
|
giving
|
|
|
```
|
|
|
```sql
|
|
|
QUERY PLAN
|
|
|
---------------------------------------------------------------
|
|
|
Insert on forward_streams (cost=0.00..0.01 rows=1 width=288)
|
... | ... | @@ -164,7 +164,7 @@ giving |
|
|
(2 rows)
|
|
|
```
|
|
|
To preview the submission of the Fusion Matlab Quicklooks to Forward Stream, we enter the following in the `psql` shell:
|
|
|
```
|
|
|
```sql
|
|
|
explain INSERT INTO forward_streams (
|
|
|
name, offset_start, offset_end, find_contexts_arguments,
|
|
|
workflow_head, workflow_targets, job_mods
|
... | ... | @@ -181,7 +181,7 @@ VALUES ( |
|
|
;
|
|
|
```
|
|
|
which outputs
|
|
|
```
|
|
|
```sql
|
|
|
QUERY PLAN
|
|
|
---------------------------------------------------------------
|
|
|
Insert on forward_streams (cost=0.00..0.01 rows=1 width=288)
|
... | ... | @@ -190,6 +190,30 @@ which outputs |
|
|
```
|
|
|
To actually submit the task, remove the `explain` keyword from the above invocation.
|
|
|
|
|
|
## Modifying a Forward Stream Instance
|
|
|
|
|
|
If we want to change the version number or some other characteristic of a forward stream instance (say `FUSION_MATLAB`)
|
|
|
```sql
|
|
|
flo3=> select * from forward_streams where name = 'FusionMatlab';
|
|
|
id | name | offset_start | offset_end | find_contexts_arguments | workflow_head | workflow_targets | workflow_download_onlies | job_mods | output_volume | num_retries | expiration
|
|
|
----+--------------+--------------+------------+-----------------------------------------------+------------------------------------+------------------------------------------------+--------------------------+-------------------------------+---------------+-------------+------------
|
|
|
43 | FusionMatlab | -4 days | 00:00:00 | "version"=>"'1.0dev3'", "satellite"=>"'snpp'" | flo.sw.fusion_matlab:FUSION_MATLAB | {flo.sw.fusion_matlab:FUSION_MATLAB;fused_l1b} | {} | "requests"=>"['Memory=8000']" | | |
|
|
|
(1 row)
|
|
|
```
|
|
|
we can preview the change by running
|
|
|
```sql
|
|
|
select find_contexts_arguments || 'version=>"''1.0dev6''"' from forward_streams where name = 'FusionMatlab';
|
|
|
```
|
|
|
When we are happy with the proposed changes we can modify the existing forward stream
|
|
|
```sql
|
|
|
update forward_streams set find_contexts_arguments = find_contexts_arguments || 'version=>"''1.0dev6''"' where name = 'FusionMatlab';
|
|
|
```
|
|
|
For the Fusion Quicklooks we similarly have
|
|
|
```sql
|
|
|
select find_contexts_arguments || 'version=>"''1.0dev4''"' from forward_streams where name = 'FusionMatlabDailyQL';
|
|
|
update forward_streams set find_contexts_arguments = find_contexts_arguments || 'version=>"''1.0dev4''"' where name = 'FusionMatlabDailyQL';
|
|
|
```
|
|
|
|
|
|
## Examining log files of failed jobs
|
|
|
The details o failed jobs can be found from
|
|
|
```
|
... | ... | |