WIP: Add "--fields" option for glance info
This is a work in progress, do not merge Based on later experience, I want to rethink my design, including if it should be integrated with stats and inspectStats.
--fields is a comma seperated list of fields to be output. By default, it's "filename,variable", which is the old output. You can use any of:
- filename
- variable
- shape - data shape of the variable
- ATTRIBUTE_NAME - any attribute associated with the variable. If missing, will be an empty string.
- stats(STAT_NAME) - any statistic as seen in glance inspectStats. The names are exactly as shown there, so "stats(min)" and "stats(finite_count)" are options.
I'd regularly wanted "shape" and the attribute "long_name" for use while exploring new data. A need to create a report of all of our output data drove generalizing the problem and adding stats(STAT_NAME).
Examples:
filename,variable
is identical to using glance info
without any --fields
option at all.
$ glance info FILE.nc
FILE.nc: Latitude
Longitude
SensorAzimuth
SensorZenith
SolarAzimuth
SolarZenith
SpaceMask
$ glance info --fields filename,variable FILE.nc
FILE.nc: Latitude
Longitude
SensorAzimuth
SensorZenith
SolarAzimuth
SolarZenith
SpaceMask
If you aren't using --parsable
, it will insert filename
at the front if not already present. This is a side effect of striving for maximum backward compatibility.
$ glance info --fields long_name,filename FILE.nc
FILE.nc: Latitude FILE.nc
Longitude FILE.nc
Sensor azimuth angle FILE.nc
Sensor zenith angle FILE.nc
Solar azimuth angle FILE.nc
Solar zenith angle FILE.nc
Space Mask FILE.nc
I use this command when exploring new files. shape
is calculated by glance. units
and long_name
attributes associated with each variable.
$ glance info --parsable --fields variable,shape,units,long_name FILE.nc
Latitude (5424, 5424) degrees_north Latitude
Longitude (5424, 5424) degrees_east Longitude
SensorAzimuth (5424, 5424) degrees Sensor azimuth angle
SensorZenith (5424, 5424) degrees Sensor zenith angle
SolarAzimuth (5424, 5424) degrees Solar azimuth angle
SolarZenith (5424, 5424) degrees Solar zenith angle
SpaceMask (5424, 5424) 1 Space Mask
For a recent report, I needed a pile of the calculated statistics. The names are the same names that appear in the statistics reports. Here's a simple example:
$ glance info --parsable --fields 'variable,stats(min),stats(max)' FILE.nc
Latitude -81.1488 81.1469
Longitude -156.195 6.19885
SensorAzimuth 0.0190026 359.981
SensorZenith 0.0151067 90.0
SolarAzimuth 0.000601057 360.0
SolarZenith 22.2221 179.96
SpaceMask 0.0 1.0
Here's the actual command I ran:
glance info --parsable --fields "filename,variable,long_name,units,shape,"\
"stats(min),stats(max),stats(mean),stats(std_val),stats(finite_count),"\
"stats(finite_fraction),stats(missing_count),stats(missing_fraction)" \
*.[nN][Cc] > variable-report.tsv
(This was migrated from https://github.com/evas-ssec/uwglance/pull/13 )