Make reports use less disk space
Reports with images can be pretty big, especially when comparing 100+ variables. A comparison of some GOES-R level 2 outputs, 104 variables total, is about 73MiB.
The bulk of the space is going to images. Here's a typical single variable:
Size (KiB) | File |
---|---|
1,452 | TOTAL |
217 | A.png |
202 | B.png |
135 | Hex.png |
88 | Diff.png |
80 | AbsDiff.png |
76 | small.A.png |
74 | small.AbsDiff.png |
73 | small.B.png |
72 | Mismatch.png |
72 | Hist.png |
70 | small.Diff.png |
64 | DensityScatter.png |
62 | small.DensityScatter.png |
60 | small.Mismatch.png |
47 | small.Hex.png |
27 | small.Hist.png |
9.6 | index.html |
These are big enough to be a nuisance to hand around. If we could shrink them, it would be helpful.
Possible techniques to make things smaller:
-
Make a command line version of the option to DO_IMAGES_ONLY_ON_FAIL_KEY. -
Don't generate small.*
images; use the originals directly and rescale in browser. It will use more browser memory, CPU, and bandwidth., but I don't think it's significant. This might save about 30% of the space. -
Make our images indexed color instead of RGBA.It looks like there isn't a way to do this conversion in matplotlib. We could probably do it with PIL but my back of envelope tests suggest it would perceptibly damage the image, so I'm not going to do it for now. -
Higher compression levels. perhapsProbably not; Going from compression 6 to 9 saves about 2%, turning on optimization saves nothing (indeed, it grows a bit!).compress_level=9, optimize=True
? (default iscompress_level=6, optimize=False
).
Comparison of compression levels:
Size (B) | Time (s) | compress_level | optimize |
---|---|---|---|
1,590,874 | 2.74 | 1 | False |
1,447,641 | 4.44 | 6 | False |
1,414,188 | 20.09 | 9 | False |
1,416,435 | 19.97 | 9 | True |