yori granule level output file size
So Paul's most recent attempt at using Yori is putting out 23 GB granule files. Yes, he is gridding a lot of stuff, by my count its 103 variables and 122 JHistos:
[steved@sipsdev 20180912-1]$ pwd /mnt/deliveredcode/deliveries/cldprop_preyori/20180912-1 [steved@sipsdev 20180912-1]$ grep name_out dist/config_v0.0.3.yml | grep -v JHisto | wc -l 103 [steved@sipsdev 20180912-1]$ grep name_out dist/config_v0.0.3.yml | grep JHisto | wc -l 122
He takes the original 447 MB L2 file, writes out a 167 MB filtered file and then when it runs through Yori it becomes a 23 GB file. Now, if I apply compression to that Yori output file it drops it to 137 MB.
In my opinion that output size of 23 GB is a little big. I'm kind of the opinion that we should have compression and chunking turned on by default. So, you might want to consider this.
Also one other thing I noticed, the n_points variables are all doubles. Is there a reason that these are not integers (possibly unsigned integers with a fill value)?