Newer
Older
# EdosL0Util
Tools for mangling EDOS L0 PDS files. This package was primarily written to
support work with EDOS L0 PDS files but can more generically be considered a
CCSDS packet file manipulation toolkit.
* API for streaming packets
* CLI tools for:
- merging multiple PDS files
- truncating files to specified time range
- inspecting PDS info
* Dumping JPSS H5 RDRs to L0 PDS files for VIIRS, CrIS, and ATMS
There is also support for creating a packet stream from Aqua L0 PDS files, but
it has not been tested much. Most functionality assumes EDOS L0 PDS files.
### Packet streaming
To obtain a simple stream of packets, with support for secondary header
parsing use the ``PacketStream`` class as an iterator.
>>> from edosl0util.stream import PacketStream
>>> stream = PacketStream(open('P1570826VIIRSSCIENCEAS15286023813501.PDS', 'rb'))
>>> stream.next()
<Packet apid=826 seqid=13072 stamp=2015-10-13 00:22:04.042798>
>>> packets = list(stream)
>>> first, last, info = stream.info()
>>> print("First:{}\nLast:{}".format(first, last))
First:2015-10-13 00:22:04.042798
Last:2015-10-13 02:04:58.107582
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
>>> print info
defaultdict(<function edosl0util.stream.<lambda>>,
{800: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
801: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
802: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
803: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
804: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
805: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
806: {'count': 59005, 'last_seqid': 2684, 'num_missing': 0},
807: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
808: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
809: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
810: {'count': 32351, 'last_seqid': 16043, 'num_missing': 0},
811: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
812: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
813: {'count': 114543, 'last_seqid': 5214, 'num_missing': 0},
814: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
815: {'count': 58994, 'last_seqid': 2673, 'num_missing': 0},
816: {'count': 58990, 'last_seqid': 2669, 'num_missing': 0},
817: {'count': 114510, 'last_seqid': 5181, 'num_missing': 0},
818: {'count': 62766, 'last_seqid': 11834, 'num_missing': 0},
819: {'count': 62766, 'last_seqid': 11834, 'num_missing': 0},
820: {'count': 62766, 'last_seqid': 11834, 'num_missing': 0},
821: {'count': 59007, 'last_seqid': 2686, 'num_missing': 0},
825: {'count': 83280, 'last_seqid': 3768, 'num_missing': 0},
826: {'count': 3471, 'last_seqid': 158, 'num_missing': 0}})
To handle ``PacketTooShort`` and ``NonSequentialSeqId`` errors you have to
iterate manually:
>>> from edosl0util.stream import PacketStream
>>> stream = PacketStream(open('P1570826VIIRSSCIENCEAS15286023813501.PDS'))
>>> while True:
...: try:
...: packet = stream.next()
...: print(packet)
...: except PacketTooShort as err:
...: print("Corrupt packet stream: {:s}".format(packet))
...: break
...: except NonConsecutiveSeqId as err:
...: print("Missing packet: {:s}".format(err))
...: except StopIteration:
...: break
### Command Line Tools
There are command line tools for merging, splitting, truncating and inspecting
L0 PDS files.
* edosl0merge
* edosl0split
* edosl0trunc
* edosl0info
* rdr2l0
### Creating L0 PDS Files from RDRs
The ``rdr2l0`` tool will dump out an RDR in an order not suitable for use as a
NASA L0 PDS file. To generate a L0 PDS file actually requires 2 steps:
```
$> rdr2l0 --diary --skipfill RNSCA-RVIRS_npp_d20151018_t2359214_e0000467_b20595_c20151019015559980487_noaa_ops.h5
$> edosl0merge -o out.PDS RNSCA-RVIRS_npp_d20151018_t2359214_e0000467_b20595_c20151019015559980487_noaa_ops.h5.science.PDS
The resulting file ``out.PDS`` will have packets sorted by time and APID with
The typical case would be to dump multiple RDR files to PDS then merge the
resulting PDS files into a 6 minute granule like so:
```
$> rdr2l0 --diary --skipfill RNSCA-RVIRS_npp_d20151018_t000*.h5
$> edosl0merge -o out.PDS --trunc_to "2015-10-18 00:00:00,2015-10-18 00:06:00" *.science.PDS
```
Which will create a proper 6 minute NASA L0 PDS file with any fill packets
filtered out.
If you have quesions or comments related to this software contact