| ... | @@ -4,13 +4,12 @@ In order to use the built in filters you need to import them in your configurati |
... | @@ -4,13 +4,12 @@ In order to use the built in filters you need to import them in your configurati |
|
|
|
|
|
|
|
`import glance.filters as filters`
|
|
`import glance.filters as filters`
|
|
|
|
|
|
|
|
###Reshaping and Reordering
|
|
<h3>Reshaping and Reordering</h3>
|
|
|
|
|
|
|
|
|
|
|
|
|
The simplest of the filter functions are the trimming functions. These remove a specified number of elements from an edge of a 2D data set. Besides the obvious use of narrowing the window of examination, they may also be used to shift data very easily in the case where the edges of your data are mostly non-finite/missing.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The simplest of the filter functions are the **trimming functions**. These **remove a specified number of elements from an edge of a 2D data set**. Besides the obvious use of narrowing the window of examination, they may also be used to shift data very easily in the case where the edges of your data are mostly non-finite/missing.
|
|
|
|
|
|
|
|
|
```
|
|
|
trim_off_of_top (data, num_elements_to_trim)
|
|
trim_off_of_top (data, num_elements_to_trim)
|
|
|
"""
|
|
"""
|
|
|
Remove num_elements_to_trim rows from the top of the data array
|
|
Remove num_elements_to_trim rows from the top of the data array
|
| ... | @@ -37,12 +36,11 @@ trim_off_of_left (data, num_elements_to_trim) |
... | @@ -37,12 +36,11 @@ trim_off_of_left (data, num_elements_to_trim) |
|
|
Remove num_elements_to_trim columns from the left side of the data array
|
|
Remove num_elements_to_trim columns from the left side of the data array
|
|
|
and return a copy of the remaining data
|
|
and return a copy of the remaining data
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
The **slicing function** is a good way to **reduce 3D data sets to 2D** ones that can be more easily visualized. At the moment the slicing function assumes that you want to slice across the third index of the array.
|
|
|
|
|
|
|
|
The slicing function is a good way to reduce 3D data sets to 2D ones that can be more easily visualized. At the moment the slicing function assumes that you want to slice across the third index of the array.
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
select_slice_from_3D_last (data, slice_index)
|
|
select_slice_from_3D_last (data, slice_index)
|
|
|
"""
|
|
"""
|
|
|
Select a slice from a 3 dimensional data set.
|
|
Select a slice from a 3 dimensional data set.
|
| ... | @@ -52,26 +50,24 @@ select_slice_from_3D_last (data, slice_index) |
... | @@ -52,26 +50,24 @@ select_slice_from_3D_last (data, slice_index) |
|
|
note: this assumes you wish to slice across the dimension that you would index
|
|
note: this assumes you wish to slice across the dimension that you would index
|
|
|
into last
|
|
into last
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The rotate_indexes_right function allows you to **reorder the indexes of a three dimensional array**. It takes an array that is indexed [a, b, c] and changes the order to [c, a, b]. You can apply the filter multiple times to achieve other orders.
|
|
|
The rotate_indexes_right function allows you to reorder the indexes of a three dimensional array. It takes an array that is indexed [a, b, c] and changes the order to [c, a, b]. You can apply the filter multiple times to achieve other orders.
|
|
|
|
|
|
|
|
|
|
Note: Not all orders can be achieved with this method. If more complex reordering filters are needed in the future please feel free to create them in a similar style.
|
|
Note: Not all orders can be achieved with this method. If more complex reordering filters are needed in the future please feel free to create them in a similar style.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
def rotate_indexes_right (data)
|
|
def rotate_indexes_right (data)
|
|
|
"""
|
|
"""
|
|
|
move the order of the indexes in the array to the right in order, taking the last one
|
|
move the order of the indexes in the array to the right in order, taking the last one
|
|
|
and putting it in the first index spot
|
|
and putting it in the first index spot
|
|
|
note: at the moment this filter only works with 3 dimentional data sets
|
|
note: at the moment this filter only works with 3 dimentional data sets
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
The reverse_2D_data_vertically and reverse_2D_data_horizontally functions allow you to **reverse the order of two dimensional data arrays** along the dimensions that Glance displays vertically or horizontally respectively.
|
|
|
|
|
|
|
|
The reverse_2D_data_vertically and reverse_2D_data_horizontally functions allow you to reverse the order of two dimensional data arrays along the dimensions that Glance displays vertically or horizontally respectively.
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def reverse_2D_data_vertically (data)
|
|
def reverse_2D_data_vertically (data)
|
|
|
"""
|
|
"""
|
|
|
Reverse two dimensional data along it's first dimension.
|
|
Reverse two dimensional data along it's first dimension.
|
| ... | @@ -85,15 +81,15 @@ def reverse_2D_data_horizontally (data) |
... | @@ -85,15 +81,15 @@ def reverse_2D_data_horizontally (data) |
|
|
For most satellite data this will result in it flipping horizontally
|
|
For most satellite data this will result in it flipping horizontally
|
|
|
when glance displays it.
|
|
when glance displays it.
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
<h3>Data Value Based Filters</h3>
|
|
|
|
|
|
|
|
Data Value Based Filters
|
|
The **sorting/flattening function** allows you to specify a set of bins and new values that you would like assigned to the values that fall into those bins. This, in essence, will allow you to reduce continuous data to a more discrete form or reduce discrete data to a smaller number of categories.
|
|
|
|
|
|
|
|
|
|
|
|
|
The sorting/flattening function allows you to specify a set of bins and new values that you would like assigned to the values that fall into those bins. This, in essence, will allow you to reduce continuous data to a more discrete form or reduce discrete data to a smaller number of categories. For example, you could reduce cloud mask data to simply "clear" and "cloudy" rather than the various possible intermediate values.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For example, you could reduce cloud mask data to simply "clear" and "cloudy" rather than the various possible intermediate values.
|
|
|
|
|
|
|
|
|
```
|
|
|
flatten_data_into_bins (data, ranges, new_values, missing_value, return_data_type)
|
|
flatten_data_into_bins (data, ranges, new_values, missing_value, return_data_type)
|
|
|
"""
|
|
"""
|
|
|
Sort the data into the given ranges. Each range should correspond to a value
|
|
Sort the data into the given ranges. Each range should correspond to a value
|
| ... | @@ -107,12 +103,11 @@ flatten_data_into_bins (data, ranges, new_values, missing_value, return_data_typ |
... | @@ -107,12 +103,11 @@ flatten_data_into_bins (data, ranges, new_values, missing_value, return_data_typ |
|
|
Also Note: If a data point is not found to fall within any of the ranges,
|
|
Also Note: If a data point is not found to fall within any of the ranges,
|
|
|
the missing_value will be filled into that spot instead.
|
|
the missing_value will be filled into that spot instead.
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
Glance also provides a function that will **extract a single bit of "mask" data** from an integer data set. This method allows you to unpack standard packed bit masks and specify the "true" and "false" values that they should be interpreted into.
|
|
|
|
|
|
|
|
Glance also provides a function that will extract a single bit of "mask" data from an integer data set. This method allows you to unpack standard packed bit masks and specify the "true" and "false" values that they should be interpreted into.
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
extract_bit_from_packed_mask (data, index_of_bit_to_extract,
|
|
extract_bit_from_packed_mask (data, index_of_bit_to_extract,
|
|
|
truth_value, false_value,
|
|
truth_value, false_value,
|
|
|
return_data_type)
|
|
return_data_type)
|
| ... | @@ -133,29 +128,24 @@ extract_bit_from_packed_mask (data, index_of_bit_to_extract, |
... | @@ -133,29 +128,24 @@ extract_bit_from_packed_mask (data, index_of_bit_to_extract, |
|
|
use this filter multiple times to extract each separately and then create
|
|
use this filter multiple times to extract each separately and then create
|
|
|
your own filter function to perform any comparisons.
|
|
your own filter function to perform any comparisons.
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
The set_to_value_between_bounds function allows you to s**et all data in an array that is non-finite or outside a given set of bounds to some inputted value**. This is useful if you need to analyze a data set while ignoring certain types of invalid (ex. out of range) data by setting it to some known value, possibly the missing value.
|
|
|
|
|
|
|
|
The set_to_value_between_bounds function allows you to set all data in an array that is non-finite or outside a given set of bounds to some inputted value. This is good if you need to analyze a data set while ignoring certain types of invalid (ex. out of range) data by setting it to some known value, possibly the missing value.
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
def set_to_value_between_bounds(data, value_to_set_to, bottom_bound_exclusive, top_bound_exclusive) :
|
|
def set_to_value_between_bounds(data, value_to_set_to, bottom_bound_exclusive, top_bound_exclusive) :
|
|
|
"""
|
|
"""
|
|
|
Wherever the data is non-finite or outside the given bounds, set it to the given value.
|
|
Wherever the data is non-finite or outside the given bounds, set it to the given value.
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
<h3>Complex Filters</h3>
|
|
|
|
|
|
|
|
Complex Filters
|
|
The filter_based_on_additional_data_set_min_max_bounds filter can be used to **remove data** from your data set (by setting it to the missing value) **based on** where data of certain values is in **a secondary "filter" data set**. For example, this filter might be useful in selecting only the winds vectors that correspond to a certain flag value or range of quality index values.
|
|
|
|
|
|
|
|
|
|
|
|
|
The filter_based_on_additional_data_set_min_max_bounds filter can be used to remove data from your data set (by setting it to the missing value) based on where data of certain values is in a secondary "filter" data set. For example, this filter might be useful in selecting only the winds vectors that correspond to a certain flag value or range of quality index values.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: This filter was originally created to be used with secondary variable dependent filters.
|
|
Note: This filter was originally created to be used with secondary variable dependent filters.
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
def filter_based_on_additional_data_set_min_max_bounds(data, filterData, missingValue=None,
|
|
def filter_based_on_additional_data_set_min_max_bounds(data, filterData, missingValue=None,
|
|
|
minOkFilterValue=None, maxOkFilterValue=None) :
|
|
minOkFilterValue=None, maxOkFilterValue=None) :
|
|
|
"""
|
|
"""
|
| ... | @@ -166,9 +156,9 @@ def filter_based_on_additional_data_set_min_max_bounds(data, filterData, missing |
... | @@ -166,9 +156,9 @@ def filter_based_on_additional_data_set_min_max_bounds(data, filterData, missing |
|
|
|
|
|
|
|
ex. this filter might be used to remove winds data that has a quality index below a certain threshold
|
|
ex. this filter might be used to remove winds data that has a quality index below a certain threshold
|
|
|
"""
|
|
"""
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The organize_ipopp_data_into_image filter **reorders IPOP Cris data into a pseudo-image for display**. The image is composed of either one selected wave number or the mean of all wave numbers at a given point. Each pixel will represent the data at one detector, in one field of regard, on one scan line. The filter description below gives more explicit details on the physical ordering of the data in the image. The purpose of the image created by this filter is primarily to give you an idea of general spatial problems and groupings of missing data in the image taken by the hardware.
|
|
|
The organize_ipopp_data_into_image filter reorders IPOP Cris data into a pseudo-image for display. The image is composed of either one selected wave number or the mean of all wave numbers at a given point. Each pixel will represent the data at one detector, in one field of regard, on one scan line. The filter description below gives more explicit details on the physical ordering of the data in the image. The purpose of the image created by this filter is primarily to give you an idea of general spatial problems and groupings of missing data in the image taken by the hardware.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| ... | @@ -217,4 +207,3 @@ Multiple filters can be chained to produce more complex behavior. |
... | @@ -217,4 +207,3 @@ Multiple filters can be chained to produce more complex behavior. |
|
|
filterFunction = (lambda data: filters.trim_off_of_top(
|
|
filterFunction = (lambda data: filters.trim_off_of_top(
|
|
|
filters.trim_off_of_right(
|
|
filters.trim_off_of_right(
|
|
|
filters.extract_bit_from_packed_mask(data, 3, (1,0), int), 2), 2)) |
|
filters.extract_bit_from_packed_mask(data, 3, (1,0), int), 2), 2)) |
|
|
|
\ No newline at end of file |
|
|
|
|
|
|
\ No newline at end of file |
|
|