UW-Glance issueshttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues2023-10-02T14:25:21Zhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/96Move Glance to python 3.102023-10-02T14:25:21ZEva SchifferMove Glance to python 3.10Move Glance to working in python 3.10. Currently Glance is tested and functioning in 3.6, but things have moved on, so the supporting software we rely on isn't all available in 3.6 anymore. This transision will probably require a lot of ...Move Glance to working in python 3.10. Currently Glance is tested and functioning in 3.6, but things have moved on, so the supporting software we rely on isn't all available in 3.6 anymore. This transision will probably require a lot of small clean up. The list below is what I know about so far and will be expanded as I investigate.
- [x] Review changes in python from 3.6 to 3.10 and make fixes in Glance as needed
- [x] Move from easy_install to a newer installation procedure (probably pip based?)
- [x] Rewrite the installation instructions on the wiki (note: include move from conda to mambaforge)
- [x] Do some minimal testing of Glance to make sure things don't break
- [x] numpy.bool causes a fatal AttributeError if you run the guiEva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/91investigate case sensitive issues with data scaling2023-05-30T15:54:39ZEva Schifferinvestigate case sensitive issues with data scalingMy notes on this are very old, but I need to double check that Glance is handling any data scaling attributes in a case-insensitive manner. I know I'm handling some attributes appropriately, but I should investigate just in case I missed...My notes on this are very old, but I need to double check that Glance is handling any data scaling attributes in a case-insensitive manner. I know I'm handling some attributes appropriately, but I should investigate just in case I missed some cases (as I didn't take detailed notes of why I was worried about this problem).Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/88Make sure summary report works properly with missing variables2022-04-07T14:57:43ZEva SchifferMake sure summary report works properly with missing variablesNow that the main report includes variables which were requested but not found when the user asks to be warned about missing variables, the summary report will need to handle that more gracefully.Now that the main report includes variables which were requested but not found when the user asks to be warned about missing variables, the summary report will need to handle that more gracefully.Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/86stats should respect configuration files2022-03-03T19:35:05ZEva Schifferstats should respect configuration filesRight now calls to stats ignore configuration files that you specify on the command line. stats should load and respect the relevant information from a provided configuration file.Right now calls to stats ignore configuration files that you specify on the command line. stats should load and respect the relevant information from a provided configuration file.Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/49Add config file processing for stats2022-03-03T19:15:05ZEva SchifferAdd config file processing for statsRight now stats is the only main function of glance that will just ignore your config file. It would be good to fix that so stats has a few more knobs.
Notes on what stuff in the config file is meaningful for stats:
- nothing in the se...Right now stats is the only main function of glance that will just ignore your config file. It would be good to fix that so stats has a few more knobs.
Notes on what stuff in the config file is meaningful for stats:
- nothing in the settings matters
- we don't have any use for the lon/lat info
- [x] we need to respect many of the default values, including
- [x] the epsilon and fill value
- [x] the various failure configurations matter if we're going to do pass fail for the return code (I'm not sure I want to do that on this ticket, but a thing to think about)
- we don't really care about any of the plot configuration stuff including the various bin/tuple and vector plotting stuff
- [x] In the setOfVariables we'll need to respect similar stuff to the settings
- [x] technical vs display names
- [x] epsilon and fill value
- [x] the failure configuration stuff
- [x] the basic and variable dependant filtering
Stuff to fix:
- [x] one input stats
- [x] handle the technical and display names correctly
- [x] respect fill value from command line or config (default or var specific)
- [x] handle basic and variable dependent filtering
- [x] two input stats
- [x] handle the technical and display names correctly
- [x] respect epsilon and fill value from the command line or config (default or var specific)
- [x] handle basic and variable dependant filtering
- [x] do pass/fail testing from command line or config when needed and change return codes based on that (from default or var specific in config)Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/85Glance crashes when opening a str variable in the GUI (or stats)2022-03-01T19:14:54ZEva SchifferGlance crashes when opening a str variable in the GUI (or stats)If you open a string type variable in the GUI or stats Glance will crash. We should handle this more gracefully.
I've attached an example file. The variable that triggers the problem is AMVChannelName.
[GOES17_ABI_2KM_MESO_2021056_194...If you open a string type variable in the GUI or stats Glance will crash. We should handle this more gracefully.
I've attached an example file. The variable that triggers the problem is AMVChannelName.
[GOES17_ABI_2KM_MESO_2021056_1942_25_WINDS_AMV_EN-14-CT.nc](/uploads/c449d0fa6d1b7e82ff1c8d1fd63e2e16/GOES17_ABI_2KM_MESO_2021056_1942_25_WINDS_AMV_EN-14-CT.nc)https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/84Make more things configurable in the config file2022-02-25T19:46:19ZEva SchifferMake more things configurable in the config fileRight now Glance is getting some things from the command line that don't have an equivalent in the config file. I want to keep the output path and input file paths command line only, but for all the others, we should just have a way to s...Right now Glance is getting some things from the command line that don't have an equivalent in the config file. I want to keep the output path and input file paths command line only, but for all the others, we should just have a way to specify them in the config file.
Make a way to set the following in the config file
- [x] the **--nolonlat** option (maybe by setting the lat_lon_info to None?)
- [x] the **--warnmissingvars** option (maybe in the settings?)
I will also need to
- [x] Change the associated documentation to describe how these new settings work
- [x] Alter or add example configuration files to demonstrate how to use these
Note: the work to handle this for the **--doPassFail** option is being handled in the ticket for the pass/fail redesign: https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/83Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/83the --doPassFail command line option is not working properly (and needs some ...2022-02-24T16:11:35ZEva Schifferthe --doPassFail command line option is not working properly (and needs some redesign)At the moment the --doPassFail command line option turns on pass/fail testing for the purposes of the return codes, but it doesn't actually control how reports generate at the moment.
On a higher level, I think I want this setting to b...At the moment the --doPassFail command line option turns on pass/fail testing for the purposes of the return codes, but it doesn't actually control how reports generate at the moment.
On a higher level, I think I want this setting to be on by default and I need to create an appropriate config file setting that will let you do the same thing.
- [x] Turn pass/fail testing on by default
- [x] Make a command line option to turn pass/fail testing off
- [x] Make the current command line option (that turns pass/fail testing on) invisible in the docs (because it won't be needed anymore) but leave it in the system so people's existing calls don't break
- [x] Make a config file setting to turn pass/fail on and off (probably in the settings dictionary)
- [x] review report and stats code to make sure it will respect the pass/fail setting for all things (returns and how variables are analyzed)
- [x] Change the documentation to reflect the new way pass/fail testing works
- [x] Update that one config file that's intended to turn pass/fail testing offEva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/75Revise reports based on Mike Walters' examples2022-02-18T21:04:52ZEva SchifferRevise reports based on Mike Walters' examplesMike Walters did some reworking of the reports locally and I think some of the stuff he did may generalize to making our reports more clear and useful.
- [x] add a command line option to turn on variables missing that were specified in...Mike Walters did some reworking of the reports locally and I think some of the stuff he did may generalize to making our reports more clear and useful.
- [x] add a command line option to turn on variables missing that were specified in the config file being an error (done in https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/17 )
- [x] add a line to the comparison results table for each variable that was requested (in the config or on the command line as appropriate) but could not be found or compared for whatever reason (only when '--warnmissingvars' is used on the command line)
- [x] add a summary line for the compared variables with how many passed and failed (+ errored)
- [x] a table based display of the variable comparison details in the "Compared Variables" section
~~- [ ] next/back/up links in the header to make quickly navigating the whole report easy~~
- [x] on the variable page, note whether or not that variable passed comparison
Here are some example reports from Mike for reference:
[index__1_.html](/uploads/51e33c6a0f143814cf1594d29169c3e9/index__1_.html)
[index__2_.html](/uploads/b4d3e5f50293a06a14373b2354f0fbf9/index__2_.html)
[index.html](/uploads/40e97ffab590924e429c322151663108/index.html)
_Mike has also requested the ability to have Glance use configurable custom report templates with the same input data as my regular templates. I'm not doing that in this issue, but this is a note to myself to reassess how difficult that's going to be once I've done the other stuff here._
Note to self: I added a '--warnmissingvars' command line option to warn about missing variables on https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/17 but I still need to add the changes to put that info in the main report page on this ticket.
Edit: I've decided not to do the next/back/up links, since a batch of reports can more easily be navigated from the new summary page.Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/66Give more basic variable information on the main page of reports2022-02-15T19:50:57ZEva SchifferGive more basic variable information on the main page of reportsOn the main page of the report, give shape information for variables and where variables have only a single value (there are a few ways, see the GUI single value display) display that value. This information is currently available in var...On the main page of the report, give shape information for variables and where variables have only a single value (there are a few ways, see the GUI single value display) display that value. This information is currently available in variable reports, but should be super easy to display clearly on the main summary page as well.
- [x] give shape information for variables on the main page of the report
- [x] where variables have only a single value display that value on the main page of the reportEva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/78rethink the info command2022-01-28T17:10:34ZEva Schifferrethink the info commandRight now the info command just gives you a list of available variables in a file. We can pretty easily organize a lot more basic information about the file contents (without actually looking at the data).
- show file info (dimensions,...Right now the info command just gives you a list of available variables in a file. We can pretty easily organize a lot more basic information about the file contents (without actually looking at the data).
- show file info (dimensions, global attributes)
- show variable info (size/type, dims, attributes)
This will need to be implemented/tested for all the types of files Glance supports:
- [x] hdf
- [x] netCDF (with and without groups)
- [x] h5
- [ ] AERI
- [x] GeoTIFFEva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/31Support for NetCDF4 Groups2022-01-27T16:39:46ZZach GriffithSupport for NetCDF4 GroupsCurrently everything is expected to be top-level variables in NetCDF files. It would be nice to be able to use this tool with files using [NetCDF4 groups](http://unidata.github.io/netcdf4-python/#section2).Currently everything is expected to be top-level variables in NetCDF files. It would be nice to be able to use this tool with files using [NetCDF4 groups](http://unidata.github.io/netcdf4-python/#section2).Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/79Fix GeoTIFF plotting in the gui2022-01-11T19:41:39ZEva SchifferFix GeoTIFF plotting in the guiAt one point Glance was able to plot GeoTIFFs in the gui as rgb images.
I think I broke that while trying to keep other errors from causing crashes when plotting. Right now if you try to plot a GeoTIFF as an rgb image, you get a messag...At one point Glance was able to plot GeoTIFFs in the gui as rgb images.
I think I broke that while trying to keep other errors from causing crashes when plotting. Right now if you try to plot a GeoTIFF as an rgb image, you get a message like:
"Two dimensional data is required for this plot type. The provided variable data is (1500, 1500, 3) shaped."Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/77If you start up the gui with a non-existent file name it will crash2021-12-16T18:14:42ZEva SchifferIf you start up the gui with a non-existent file name it will crashNormally if you give the gui command line a file name it will try to load it. If you give it a file name that doesn't exist it will correctly report that it could not be opened because it doesn't exist, but then try to access stuff from ...Normally if you give the gui command line a file name it will try to load it. If you give it a file name that doesn't exist it will correctly report that it could not be opened because it doesn't exist, but then try to access stuff from the file anyway and crash. It should just load up the gui with no file loaded at all.Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/65Treat units of "1" as not having units for the purposes of plot displays2021-12-10T20:10:11ZEva SchifferTreat units of "1" as not having units for the purposes of plot displaysWhen labeling the color bar with units, if the units are 1, treat that as if there were no units by not putting a unit label on the plot displays.When labeling the color bar with units, if the units are 1, treat that as if there were no units by not putting a unit label on the plot displays.https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/64Double check that the range colors and labels are working properly2021-12-09T22:03:57ZEva SchifferDouble check that the range colors and labels are working properlyI have some old reports that the range colors and labels that are configurable via the config file are not working anymore. Investigate if there is actually a problem (try cloud mask maybe?).I have some old reports that the range colors and labels that are configurable via the config file are not working anymore. Investigate if there is actually a problem (try cloud mask maybe?).Eva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/54Trim excess white borders on generated graphs2021-12-07T19:57:44ZAlan De SmetTrim excess white borders on generated graphsGenerated graphs have lots of whitespace around them because matplotlib loves whitespace. However, I'm less of a fan of lots of whitespace, especially when I'm trying to pack a bunch of charts into a small space to easily skim them. For ...Generated graphs have lots of whitespace around them because matplotlib loves whitespace. However, I'm less of a fan of lots of whitespace, especially when I'm trying to pack a bunch of charts into a small space to easily skim them. For example:
![before](/uploads/24e78bd91a528a1722660fd7fa618147/before.png)
Less margin would be better. Perhaps something like:
![after](/uploads/d6baa0efbf50a8e941970d3d5dd3c81d/after.png)
One possible solution is to use [matplotlib.pyplot.savefig](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html)'s `bbox_inches` and optionally `pad_inches`. Setting `bbox_inches='tight'` will cause matplotlib to remove as much border as possible, then re-add pad_inches (which defaults to 0.1). This is what generated the second image above. You can trim even tighter, all the way down to `pad_inches=0`, although that gets very cramped feeling when looking at the image by itself.
So in pyglance/glance/plot.py in _handle_fig_creation_task around line 60, you could change
```
figure_obj.savefig(os.path.join(outputPath, fullFigName), dpi=fullDPI)
```
to
```
figure_obj.savefig(os.path.join(outputPath, fullFigName), dpi=fullDPI, bbox_inches='tight', pad_inches=0.1)
```
There are other calls to savefig in Glance; I don't know if they should have similar arguments applied.
One potential down side: the resulting images will not have a consistent size! Long labels on the left side, the presence or absence of a color bar on the right, multiple lines of title, and other things can change the size of the "core" chart. So if consistent sizes are important... this might be a problem.https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/72Update how the documentation is shown on reports2021-12-01T20:48:14ZEva SchifferUpdate how the documentation is shown on reportsVariable reports have asterisks that link to the glossary. Improvements: A small "?", probably superscripted, it a more common idiom. Ideally the link should use an #anchor to link directly to the exact glossary entry. A wee bit of ja...Variable reports have asterisks that link to the glossary. Improvements: A small "?", probably superscripted, it a more common idiom. Ideally the link should use an #anchor to link directly to the exact glossary entry. A wee bit of javascript (or maybe just pure CSS) could also make it so that hovering or clicking the "?" would cause the glossary entry to appear inline.
Note: whatever we do here we will need to make sure we don't break the GUI display of stats in the process.
- [x] change the link to be a small superscripted "?"
- [x] #anchor directly to the glossary entry for the term
- [x] create a tooltip with the definition over the linkEva SchifferEva Schifferhttps://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/71Update doc page for comparison reports2021-12-01T18:11:46ZEva SchifferUpdate doc page for comparison reportsWe've moved to a tabular format for the statistics and this changed some of the statistic names. For example, we used to have "median_a" and "median_b" but now we only have "median" with entries for A and B.
The doc page has the old st...We've moved to a tabular format for the statistics and this changed some of the statistic names. For example, we used to have "median_a" and "median_b" but now we only have "median" with entries for A and B.
The doc page has the old stat names and descriptions, so it needs to be updated to the new ones.https://gitlab.ssec.wisc.edu/evas/UW-Glance/-/issues/73When histogram creation fails for single value variables the report still dis...2021-11-23T18:01:47ZEva SchifferWhen histogram creation fails for single value variables the report still displays a broken imageWhen I create an inspection report for a single value variable it can't make a histogram, which is fine. What's not fine is that the report has a broken image link in it, rather than just no image at all.When I create an inspection report for a single value variable it can't make a histogram, which is fine. What's not fine is that the report has a broken image link in it, rather than just no image at all.Eva SchifferEva Schiffer